Welcome to the forum!
It would help to know more about your PC, but my assumption is that the CD player input is being fed through sound processing on your motherboard (as opposed to being directly passed through to your speakers). If that's the case, then a soundcard will probably make a noticeable improvement. Integrated audio/video has improved a lot, so my experience may be too outdated - but I noticed a huge improvement in sound when I upgraded to a soundcard about ten years ago. Others here know more about modern integrated capabilities, so I'm just trying to get the ball rolling on your question.
So, unless I'm wrong, I'd look into getting a soundcard. Also, does your PC have a CD drive? If so, you could use that or (probably even better) rip the CDs to a hard drive and play them from there (much quieter than having a CD drive spinning).
There are also some nice powered speakers that you can get for a PC or home stereo system. Others here know more about them, but I think that you could get some nice powered speakers, a sound card, and a CD drive for under £300.