Store bought CDs are not the same as data CDs with MP3s, or other types of CDs you may have, but recorded CDs (CD-Rs) should play back if properly finalized.
Yes, all discs should play back fine, and you just need to look at the specifications for the player you are looking at and it should tell you all the different formats that a specific player is compatible with.
The absolute minimum should include: Blu-ray Disc, DVD Video, and commercial CDs.
But, the better players have capabilities far beyond those three basics.