True, but a miswired GFCI is nothing more than an expensive socket. And they can be, electrically speaking, noisy. That's why I'd rather have my audio equipment wired correctly and passing through a fused surge protector instead of through a GFCI.
(How many people have gone around their homes with a plug-in circuit checker? Not many, I suspect. When I went around my home, I found six miswired sockets and two GFCIs that were wired in series (which isn't dangerous but is a pure waste of money and an unnecessary complexity which offers no additional protection whatsoever).)
The other issue is the grounded-to-double insulated problem (via strapping or even an unbalanced cable). If the chassis/case for a double insulated device is energized, it may not loop back to ground. Why? Because it is electrically isolated from the "mains", to use a concise and appropriate British term.
It WILL try to find the best ground, if somebody touches it... in which case they'd be part of a ground loop that, until that exact moment in time, didn't exist. (A ground loop occurs when electrons find ground through an alternate path. It is in no way restricted to instances where this creates noise, although this is how most audio folks use it.)
Now this doesn't happen frequently because things are engineered almost to the point of being idiot proof... unless the idiot in question is holding onto a hot water radiator with one hand and touching poorly configured gear with another. But accidents do happen... not necessarily through idiocy but through a series of relatively innocuous events that end in tragedy. A great example comes to mind, but that's another story for another day...
