You are far more likely to damage speakers from too little power versus too much.
The reason is straightforward ... with lesser power, you are closer to clipping at any room Sound Pressure Level (SPL) than with a bigger amp. Clipping destroys tweeters easily and can damage any driver.
Assuming you can recognize the sound of distortion, you will naturally turn down the power when it's encountered, which eliminates the problem of clipping.
The only other time where speaker damage is common is when you are trying to fill a large area (perhaps outdoors, or a hall of some kind) and the overall SPL might exceed the natural limits. In an ordinary home setting, on the other hand, this situation is unlikely.
The reason loudspeaker manufacturers offer power recommendations is to help you size the amp to the speakers in a normal home environment. Basically the low end is how much power you should have, for a reasonable volume level, and the high end is how much the speaker is likely to need to play at it's full potential volume, again in a normal home environment. It isn't a strict limit on power, simply because having more than that amount reduces the risk of clipping.
Now, it must be said, that there are people who don't recognize the sound of an amplifier entering distortion. They are likely to overdrive the speakers because the instinct to turn it down doesn't come to them.
I've examined hundreds of speakers for warranty evaluation when I was in the business (you cut them apart and examine the voice coil), and I can confidently say that in nearly 100% of cases, it was clipping that destroyed the drivers.
It should be noted that a drastic reduction of volume below the clipping level isn't necessary; just turn it down a bit so it's no longer distorting. There is little difference in overall level from the clipping point to zero clipping, so your enjoyment of the system should not be drastically affected.
If you are one who likes very loud music, you should choose speakers that can offer that level (high SPL capability) with ideally high efficiency (plays louder with the same power) than typical home speakers. There really is no way around that, even if it means the overall Sound Quality (SQ) doesn't equal the less efficient speaker you may like; it's a trade off that must be made.
In the end, I would not worry about it too much, if at all, if I were you. Use some common sense and everything should be fine.