So you checked the polarities and they are good, and you've narrowed it down to the sub. In that case, I would definitely try to us RCA for both next. Thing is, if you want to really compare apple to apple whether then you want to in fact do what you can, to get as close to apple to apple as possible. We all know there is a gain and SN difference between balanced and single ended. I do not believe that would make a difference but let's eliminate another variable.
Another point is that you should not rely on the volume position. Volume matching using a SPL meter is a simple and practical way to do it. Ideally, if you have a good voltmeter, it is preferred, but again, a spl meter is good for all intents and purposes.
If you have done both things I mentioned above, your wife, or just a friend you is willing to participate, would not hear that "warm" thing. That, I dare say (let me apologize to you in advance now) it is the hearsay thing from the long past that got embedded in people's head and managed to generate expectation bias 90 out of 100 times. The other times would be due to something real, such as the heavier low to mid bass for one set up that may in turn be due to imperfect set up for comparisons and you have just experienced that yourself.
I would say there is nothing wrong "as such". The PM8005 does not have subwoofer output so I assume you have to use the preout. The P5 has dedicated subwoofer outputs, complete with crossover switches for both the preouts and the subouts and then it has an independent subwoofer level control knob on the front panel.
So you can see that if use the P5's subout, the gain structure between it and the Marantz PM8005 could be quite different, and compounded if you did use the crossover switches when doing you A/B comparisons.
By the way, now that you have eliminated the subwoofer (a necessary step), please make sure you turn off the P5's high pass crossover switch for the mains so that your Polks are getting full range when hooked up to the ATI power amp.
