Clipping is absolutely still part of the problem! You are correct about the average power, buuut........
The average power of a clipped signal will be more than the average power delivered by the same unclipped signal.
graphic of a sine wave:
Measuring the Sine Wave
graphic of sine and square wave
Redirect Notice
So, the Root Cause is too much average power, but the Failure Mode can be a clipped signal (allowing for too much avg power).
It's definitely multi-factorial. The clipped power (double the rated power) could damage the speaker just as much as too much unclipped power could damage the speaker.
Let's say the limit of the woofer is 200W.
If we use a 50W amp and it clips, 100W of power would reach the woofer. Because this power is way below the limit, it doesn't matter if the signal is clipped. As long as it is below the limit, the woofer will not be damaged even if the signal is clipped.
If we use a 300W amp and the volume is crazy, 300 watts of unclipped power would reach the woofer. Because it exceeds the limit, the woofer would be damaged even if the signal was never clipped.
Now if we use a 150W amp and it clipped, 300 watts of clipped power would reach the woofer and damage the woofer too.
So the limiters can help prevent the clipping, which does help. But it does not prevent excessive unclipped power from reaching the speaker.
IOW, even if the amp has limiters, you could still damage the speakers if you use crazy volume and a powerful amp.
Bottom line, as all of us have said, don't go crazy on the amp or the volume. Most people would consider 200-300W amps as normal. If the speaker is rated for 200W, I don't think it's wrong to use a 200W amp.
If the speaker is rated for 200W max and you use a 300W amp, it is most likely okay too even if it is not "recommended" by the speaker engineer.
If the speaker is rated for 200W, I don't think it is any safer to use a 300W amp than a 200W amp.
So I think the statement "It is safer to use a more powerful amp" is less true.
I think the statement "It is safer to use an amp that is recommended by the speaker engineer" is more true.
Naturally, it is safer to use sensible safe volume period.