since the volume settings don't affect this, I'd say it's the residual noise in the power amp stage of your receiver. All amps produce this to some level or another.
If you have to go out of your way to hear it, I wouldn't worry about it. In this hobby, if you look hard enough for imprefections, you'll surely find some. Enjoy the illusion of perfection. you'll be better off.