If you connect everything to the receiver, it becomes a convenient switcher. Audio follows Video, so whenever you change the source on the receiver, the audio changes automatically too. You only need 1 video connection to the tv and never have to change the tv's input.
The only argument for connecting directly to the tv is the idea that a direct connection is 'better'; ie fewer devices in the signal path should result in a better picture. In practice, it's a non-issue as receivers just pass the video signal through and don't affect the picture quality. You'll just have to try it and see if you can detect a difference.