We have been down this road many times before.
Power is voltage X amps. So the 8 ohm drive needs half the current and twice the voltage of the 4 ohm load. So the four ohm load is double the current.
That double current load has to pass though the output devices. These devices all have a resistance and it will be pretty much the same whether 4 or 8 ohm
Now this is were the problem is you have overlooked. The heating effect of that current is the square of the current times the resistance. So heating goes up by the square of the current and has to be dissipated from the output devices. Heat kills.
Just in case you think that upping the impedance would be a good idea, you are wrong again. This means increasing the voltage. High voltage and semi conductors is a bad idea, because those nasty high voltages spark and discharge through the semi conductor material and makes holes in it, destroying the device.
So there you have it.