So you assume that the four screws going from the back of the speaker through the box and into the baffle plastic are more like reinforcement and not something that does the entire job of holding the baffle? I'll try to learn as much as possible before doing anything. I wrote to the KEF company, I'm still waiting for the response.
I mean something different that what I think you've said. The screws on the front hold the driver to the baffle. I don't expect to see any other attachment. KEF will certainly know.
Once you pull out the trim ring from the front of the driver, you should see 4, 6, or 8 screws. Probably 4. They alone attach the driver frame to the front of the baffle. The diagrams I saw don't clearly show what type of screwdriver tip is required. It may be Phillips head, hexagonal, Torx, or other type.
See this link for a list & diagram of different types. Use the correct tip for the job, but don't buy anything before you see what you need.
Remove the screws carefully. Take care to never allow the tip to slip out of the screw head and puncture the woofer cone. (Trust me, I have firsthand experience

.) Square, Robertson, Hex, or Torx type screw head are good because those tips cannot slip out. Be most careful with slot drive tips. If you need Phillips head, my sympathy. (They were designed by the Devil.)
You may have to pry the woofer frame loose from the rubber gasket behind it. Do that carefully with a thin bladed flat tool, like a butter knife.
After that, the woofer should come out easily. Study the wire colors and where each wire is attached. Do not rely on your memory – write it down. (see above comment about firsthand experience).
Maaan, are these some gray days for me. One should never punch above his weight, it's too much of a tragedy when it fails. I'm posting this as a tip of the day.
Don't despair. It can be repaired. If the finish does get scratched in the process, it can be filled or polished out. Any good furniture finish repair guy can do that.