With Amazon, most of what you will buy now will reside on the cloud. That way it doesn't matter what device you are on, you just log in and have access to it. Currently that means PC, tablet, phone, etc... for now though; meaning no Blu-ray players, PS3 or TVs that I have seen have this yet, because it is not the same as Amazon VOD AFAIK. You can either stream media from their cloud or download it to the device, which I am finding a nice feature. I buy an album and the whole thing is on the cloud, but I typically don't download the whole album to my phone.
For the free Kindle books I think you have to be an Amazon Prime member, but that includes free (select, in a similar fashion to the books) movie streaming as well.
Just because manufacturers stop producing as many CDs does not mean the format will go away. Vinyl is still around and doing quite well how many years later? CDs will be around for a while. Some albums that I want to buy are not available in MP3 on Amazon as well, so there will still be cases like that where I will have to get a disc.