OK, so, if I’m understanding you correctly, if I boost the level for each input while matching (as close as I can) each one, then set individual speaker levels, I’m OK?
Yes Craig, you're OK.
But it is almost impossible to level match each source, because it depends of the particular recording playing from that source. Comprende?
Usually, you want to perhaps boost the Phono and Tape Deck sources, as compared to others.
And your individual channel levels are set for each source equally, unless your receiver or pre/pro has individual source's channel levels, or by surround audio modes, like the Denon receivers.
Either way Craig, you're fine to your heart content to experiment with all these different source and channel levels, without any ill effects.*

* The only thing that could happen, is that you loose a bit of Dynamic from the Master Volume level (instead of going all the way to let's say +18db, it could perhaps only go to +12db maximum). But this is not a negative on sound quality itself.