In most cases, when selecting both A and B speakers, the receiver is wiring them in parallel (there are a few receivers that wire them in series; if when only one pair is hooked up and you select both A and B you get no sound, that is a good indication that one has a unit that hooks them up in series). Them being in parallel means that if they are both identical sets of speakers, the amplifier is being presented with an impedance of exactly half of just having one set of speakers hooked up. So it can harm the amplifier, if the resulting impedance is too low. Consequently, normally, next to the speaker terminals, it will tell you that the minimum impedance that you can use for both A and B is twice what the minimum impedance is that you can use for either A or B.
So, it really depends on what speakers you are hooking up to it, and also how loud you play it (as playing it louder makes greater demands on the amplifier).
There is nothing inherently wrong with using both at the same time, provided one pays attention to what the requirements are for running them at the same time.