Don't overlook the issue of cooling. When a speaker lists a maximum wattage, it's about more than (if you exceed this for a moment, you blow the speaker). Sustained output generates heat that needs to be radiated. Overheating can cause all sorts of speaker seizing.
The general advice I've seen regarding over-driving, is that there's more danger of frying a speaker by putting too little amp on it and clipping than there is of putting too much amp on it and over-driving.
Keep the volume sensible for the size of the speaker (try to listen to 50w speakers from across a 3,000 ft^2 house would likely be inappropriate) and I suspect you'll be fine.