I have speakers with magnetically attached grilles, knob & socket attached grilles, and no grilles at all.
The knob & socket equipped speakers are 1988-1991 era Hafler 300 floor-standing speakers and everything is still intact. They can be very reliable, though they don't take abuse. It's plastic.
I would just replace them all with new.
If I was going the magnet route, I would embed a neodymium magnet on just one side and a steel slug on the other. That way there's no pole inversion issues yet extremely good grip.