Speaker watts and amp watts are a pretty loosey-goosey relationship.
A speaker can handle more watts than it's specced for and, likewise, it doesn't have to be fed it's maximum wattage to provide satisfyingly loud sound. In fact, it's rare that a speaker will be called upon to handle more than a fraction of it's rated power. Most listening is done with only a few watts, generally less then five.
But, if you are in the habit of cranking it unmercifully, you should be aware that you could overdrive a weak amplifier into clipping, not a good thing for either the amp or the speakers.
But, one good ting is that the amp (and the speakers) generally give audible warning before damage occurs. They start to sound harsh, gritty or just plain bad. These are signs that one of them doesn't like the levels at which it's being played.
The solution is to turn it down immediately. If you follow one simple rule, they should live to see another day.
That rule is: If it starts to sound bad, turn it down NOW!!!
This rule applies to both using a high powered amp on low powered speakers and a low powered amp on speakers that can absorb gobs of power.