Well, how broken is your old HTiB exactly? Is the DVD/BR player broken or is it the sub or one or more speakers? Because you might be able to 'recycle' some components.
This way, you could for example invest in a good/better separate receiver and two good fronts and reuse the other components from your old HTiB. You'll still have surround yet are able to redirect more money towards specific components, leading to a higher quality of those components, leading to you having to spend less money in the future (because for instance you don't have to upgrade the front speakers anymore). When on a budget, try to be creative
If this isn't possible or you don't want it, a HTiB with a 'real' reciever rather than a DVD/BR player to drive the speakers and incrementally upgrading is your best bet. Or if you really don't mind losing the 5.1 you could get a receiver + 2.1
Some other (random) remarks:
- What on earth are you still doing using composite connections? Unless you're still gaming on your NES there's no valid excuse really
. Almost all devices have component connections and it's far superior to composite. Upscaling composite is pretty useless as well, because there's not enough detail to begin with. It's not just about resolution, it's also about detail in the original picture. Also, upscaling is generally not found in the lower-end receivers.
- You going to have a very hard time finding a receiver with an optical out. You're better off connecting it directly to the source. This will probably be your only options, apart from routing the audio though your receiver's pre-outs, getting it through an ADC, encoding it on-the-fly to DD5.1 with Dolby Digital Live and then connecting your headphones there.