You have to use the speaker settings in the player, not the receiver. On the receiver side you likely have settings that adjust the levels on the multichannel input, but the size, delays, etc... are set only in the player when using analog.
Also, DVD-A and SACD are different than movies, so if you try a movie via anlog and it works fine, then you will need to make some adjustments for hirez audio vs movies. Movie audio formats have a +10dB offset built into them on the LFE channel, something that SACD and DVD-A lack. So if your player doesn't have a setting to account for this, and many do not, you will have to add +10dB when you listen to them either in the player or the receiver.