First let me say that the auto-setup probably got real close to what you want. I was just saying that you could verify it for yourself if you want.
Generally you want to calibrate to Dolby Reference Level which is 85 dB SPL at the listening position. This is when the test signal is at -20 dB. So 85+20 = 105 dB peaks (plus up to +10 dB more for the LFE channel). That is very loud and you will probably actually listen at lower levels - at least I do. Of course you can calibrate to any level you desire - what matters most is that each channel produces the same level (within a decibel or two, it's very difficult to get each exactly the same).
The catch is that you need to know the level of the test tones you are using. Receiver test tones are usually at -30 dB, so to achieve Dolby Ref Level you would calibrate so the meter reads 75 dB (75+30=105 dB peak).
I usually get it backwards, so don't quote me on this, but I think the AVIA setup disc is at -30 and DVE is at -20 dB. So use 75 dB if using AVIA and 85 dB if using DVE.
Now for the volume setting of the receiver:
It depends on the receiver. Some of them automatically increase the volume to the reference level when you use the internal test tones and thus the setting of the knob is irrelevant. Others do not. For example, my Onkyo 502 does not, but the 702 does.
If the receiver does not automatically set the volume or you are using AVIA or DVE instead of the internal test tones, you can set it wherever you like but 0 dB (if the display is relative; eg. -80 - +10) is the standard. If the receiver uses absolute display (0-80) then you should pick a point about 80% of max. I used 60 for the tx-sr502.
- Set the volume
- Start the test tones
- Hold the meter about ear level at your seating position and pointing straight up. Better yet, put it on a tripod and step away from it so your body doesn't get in the way.
- Cycle thru the channels with the remote and adjust the levels until the meter reads the number you are looking for. Do that for each channel until they all read about the same.
Now if you chose 0dB on the receiver, that is your 'reference level' and when the volume knob is in that position you will get the reference SPL at your listening position. If you turn the volume knob to -10, you will be listening at 10 dB below reference. I doubt you can stand true reference level for very long - I know I can't.
Many will use a lower value - the key point again is that all the channels are balanced at the same level, whatever that level is.
That's the quick and dirty overview of the process. I bet the Denon auto-setup got it pretty close anyway, with the exception of setting the mains to large and (possibly) the sub not being quite right.