If I had a source selector I'd try it. I'd try it because I'm willing to put in the work and research to determine if something will actually work.
In what setting or situation would this possibly be beneficial? You've mentioned "the average user" a few times. How many "average users" are gonna go get a channel selector, buy additional cable and rearrange their whole setup to do this? Half the time we can't get someone to point their speaker in the right direction, let alone complicate a setup like that.
So far, close as I can tell, it might benefit anyone off to the side, but degrades for the rest.