Glad to help, although I am pretty low on the totem pole of experience and knowledge compared to alot of people here. If you need more info you may want to try throwing some questions into other areas (the General AV or Amps, pre-pros & Receivers). More people keep their eyes on those ones.
For the sub, yes RCA to RCA is the best connection. This is assuming the sub has a 'sub in' or 'xover bypass' or something similar type of RCA connection. When you use this connection the sub simply plays what the receiver tells it. The bass management, volume, crossover, etc, is all handled by the receiver. This is why you want to use the sub out connection from the receiver. Take a look in the subwoofer section. You can get more info about how to set everything up within the recevier. I am not that familiar with you receiver so I don't know how to properly set everything up.
For 18 feet, yes 14g is fine. Some people run 10 or 12 because they want to make sure they don't degrade the signal, but this is overkill.
Running optical out from the TV to the receiver should get you the best audio you can, assuming the TV is set properly. Another option is to also run a digital coax from the DVR and a toslink from the 360 to the receiver. I am not sure if your DVR can send audio to the TV via HDMI and send audio via the digital coax at the same time. Most likely it can.
Your receiver is fine for now. All you are missing is the ability to decode the newer formats (7.1 as well as the two lossless HD version) and the auto setup functions. There are other small changes, but these two are the main ones. If you can live without these there is no need to update. Newer receivers also work as video scalars and switches. Most TVs do as good or better at handling video, so this is a nice to have but not needed.