Originally I ripped my CD collection onto my player, then I've just been adding all of the CDs and tracks I get sent to me by various bands who want played on the podcast.
For free music, try sites like Jamendo - none of the artists are in the mainstream and most are independent & unsigned, but there's some amazing stuff on there if you take the time to find it.
Subscribe to a few independent music podcasts too - plenty of free music to listen to on a regular basis!
I'm also paying for Spotify Premium at the moment - means no more annoying ads when listening to the music, plus I can use my phone as a portable player to listen to Spotify. Rather than incur data charges, you can 'download' music for listening to offline. Of course, the music's only available offline for as long as you stay a subscriber....