Using all four speakers will definitely make a difference but the one thing to remember- the power output will be shared by all speakers, so it's not as if each speaker is receiving the full power of the channels. Also, the Yamaha specs show that if the speaker load is 4 Ohms, power output is supposed to be 120W/channel.
I used a little Pioneer receiver in the mid-'70s with the original speakers in the system as well as the speakers from an old RCA phonograph system and it definitely helped even though the Pioneer was able to to put out a measly 13W.channel at 1% THD but I wasn't cranking the volume control fully.
However, if you find that you need to set the volume control past 12:00 (the indicator mark is straight up) you don't have enough power because full output is frequently reached at/near 12:00, not full rotation. Operating at such high output is dangerous, though- 100W with a speaker that's 99dB@2.83VAC sensitivity will produce more than 119dB in a moderately reflective room and the second speaker & channel will add more than 6dB to the output. This results in more than 125dB and the OSHA Noise regulation chart shows that exposure to 115dB should be limited to 1/4 hour or less, per day and 125dB is the same as increasing the power output by ten times that of 115dB.
We get one set of ears and once the damage is done, ears don't recover and normal hearing won't return.
www.osha.gov