If you're pushing 50 watts into your speakers, you're either
A. Running Some Old Soundlab Electrostat speakes
B. Playing your speakers so loud that you'd have a hard time hearing someone standing next to you outside your condo.
At any moderate to low listening level, you're probably going to be seeing 5 watt PEAKS. I have an old Yamaha CR-1020 with VU meters. I have it hooked up to some Polk Monitor 30 bookshevles (which don't sound half bad) and I cranked it up to some fairly loud levels. It barely peaked above 5 watts, and that was rare.
If you never plan on rocking out with your speakers, then you don't need a beefier amp. Just spend your money on one with good, clean outputs and low distortion.