An impedence matching speaker selector should only lower the volume to the main zone when there are other zones in use. It completely messes with any surround settings when it is in use, and is not really the way I would go.
This is, in my opinion, a horrible way to run multiple zones of audio. You can, instead, pick up an external amplifier, and run one of the record loops out of the recevier (or a completely separate source) into the amplifier, then run that into the speaker selector.
External amps are available for under $100 all the time on eBay and will have no problem running at least a couple of pairs of speakers with good volume.