you can use the THX optimizer on the Star Wars trilogy or other THX-certified DVD's (like King Arthur). it has a sub transition test to help you tell if the settings are a good match.
Does the 5760 have YPAO ? When I did the auto set-up with a Velodyne sub on the RX-V2500, I set the volume on the sub to 12 oclock, and the crossover on the sub all the way up. Once the set-up was done, I cheated and added a splitter on the sub end of the cable to get just a little more boost.
the phase selector can be used for two things... if the sub is behind you, you set the phase to reverse, so it fires opposite the mains. if the sub is in front of you, you want it to fire with the mains. you can also use it in a two-sub set-up if you have some bass cancelling. set one sub to one phase and the second to the other to eliminate the hole in the sound...