If wired properly, then red and white are the right speaker in each room and white and green are the left. White is 'red' and 'green' is black.
I would not use the Bluetooth device, though that's a perfectly acceptable source. Instead, I would look for a streaming product/device which uses wi-fi. This way, you can go anywhere in your home and things will work and it will not have Bluetooth dropouts. The Wiim Pro I believe is a 'cheap' solution which can do this nicely and is a lot less than a Sonos Port or a Bluesound Node device.
I expect your speaker selector isn't powerful enough to run the amplifier you have listed, but you need to triple check specifications. As well, I'm not sure if you got a speaker selector with integrated volume control, or if you have volume controls in each room to adjust audio volume levels, in which case it may be okay.
It would be helpful to include brand names along with specific model numbers and a link to each page so we can review the exact product you purchased.
Either way, it sounds like things are running properly. It would be good for you to pull a single set of speakers in each room to double check that red/black and white/green were used for the speakers, but that is industry standard.
Need a bit more information overall, but you aren't terribly far off it sounds like except you may have ZERO volume control in each room at this point.