The first thing to do is slow down. Go to various audio shops and take a look at what is available. Listen to a variety of speakers, making sure that you listen to a variety of types of speakers (e.g., ribbon tweeters, planar, etc.), with a variety of types of musics (all of the types you like). The more you know what you want, the less likely you will make a mistake, and therefore you will waste less money and probably end up with a better system. And before you buy a particular model, read about its performance and reliability if you can find such information (though beware of purely subjective reviews which often are based upon looks, price, reputation, exclusivity, and weight of the equipment, rather than sound).
Now, for your specific questions:
1) The Yamaha RX-V2600 you mention in your second post is an excellent choice. However, its upconversion is not stellar:
http://www.audioholics.com/productreviews/avhardware/YamahaRXV2600receiver1.php
Therefore, you are more likely to have better results if you choose a better upconverting DVD player. To pay for that, you could go with the RX-V2500 refurbished from
www.ubid.com for $439 plus shipping (so the total is likely to be less than $500 if you live in the U.S.), which many say is a good company from which to buy (I purchased my computer monitor from them years ago, and had no troubles). The RX-V2500 is not quite as good as the new RX-V2600, but it is not too far off, and is far cheaper, leaving you with more than enough for a well respected DVD player, such as the Oppo OPDV971H which sells for around $200:
http://www.oppodigital.com/
You need to have a digital video input on your TV to take advantage of its upconversion, either DVI or HDMI.
2 & 3) Bookshelf speakers are viable for surround, and for the front, too, if one has a subwoofer for the bass. Modern surround receivers can divert the deep bass from the various channels to the subwoofer output, so solid bass down to 80 Hz or a little lower is all that is typically necessary. You should try to have all of your speakers match each other. This is especially important for the front three speakers, but it matters all around to some degree. This is a good idea because when watching a movie, it is good if when something seems to go from the left to the center to the right channel, it is good if its tonal quality stays the same as it does so. Otherwise, the illusion will be broken, and the fact that you are listening to speakers that do not sound the same can be obvious. So, if you love your main speakers and do not plan on parting with them, I suggest you try to buy speakers that match them. But, before you do that, make sure you listen to a variety of other speakers first, as you may change your mind, and it will be better to change your mind before rather than after you start spending money.
Regarding the need for a subwoofer, if you are happy with the bass from your main speakers, you do not need a subwoofer. However, I suggest you go and listen to some systems with subwoofers and see if they change your mind. You are right that subwoofers that go really deep tend to be expensive, though you can get bass down to just below 20 Hz @ -3dB for about $500 delivered from SVS. They also make a subwoofer that can go down to about 12 Hz @ -3 dB for less than $1000.
This brings me to something that many people seem to forget. Speakers make the most difference in sound quality of any piece of equipment you can buy. The difference between receivers is generally slight in comparison, though, of course, convenience features can matter, and some are more powerful than others, which can matter, depending upon the speakers one is using. Anyway, I recommend that you spend MORE on speakers than on the rest of the audio system, if what you really care about is the sound. Many people buy an expensive receiver and hook up lowly speakers, and what they get are great features and unimpressive sound. If they bought a lower receiver and better speakers, they would have better sound. This, by the way, also is in keeping with my recommendation above, as the Yamaha RX-V2500 is a fine receiver with many features, and is likely all one will need (though you need to find out what features you require for yourself, and judge the matter for yourself, as it is your money and you will be the one living with the consequences of your choices).
