Just registered to chime in.
I ordered a P5 and A23 too, was looking at replacing my Sony TA-A1ES which I think lacks the bottom end drive for my KEF LS50...
I too have issue on the volume pot on the P5. The problem was not noise, but unbalanced L/R. If the pot is set to very low level (below 9 o'clock) it's clearly that the left channel is louder than right channel. Confirmed with both the headphone output and the pre-out connected to the A23 (balanced and single-ended); of course with different source - O2 DAC (single ended), HAP-Z1ES (balanced), NS3100ES (single ended) and USB. Double checked with reversing the wiring (on A23, and on my MDR-Z7 headphone) and the balance swapped to the other side instantly - confirmed.
Then, I asked for an exchange and even though it's a brand new one (the previous one was demo) it still exhibit the same issue.
The L/R unbalance is prominent in low volume level, and it continues all the way to middle level (12 o'clock position) with lesser distinction.
In any case, with that kind of "unbalance" thing happening on a pre-amp, it's not acceptable in any case. The P3 looks to have an electronic volume control and I might try my luck on a used P3 instead.
And I don't dare to try their Integrated Amp given that experience and OP's experience. Maybe the P7 is a better one (seems to have electronic volume control too) but I don't need 7 channels since I use an Onkyo Pre/Pro with a 2-channel preamp inline (Cary Audio SP-100) that connects to a Krell Evo 302 (front) and Parasound 5125 (for rear, center).
Looks like I might be just keeping the Sony TA-A1ES until I can find a used Bryston integrated or something with more oomph.
Or... go DIY and get a matching Goldpoint stepped attenuator to replace that sloppy volume pot.