The litte part thats pushed in is called the dust cap, Pushing them in generally does not affect sound quality according to many people. You can pull them back out with two main methods, The first is using a vacuume. You basically put the vacumme up to it and suck out the dent. This can be kind of dangerous if your not careful. The secone method commonly used is tape, Here is a video of how to do it.
I probably would try and pull it out, yet tell your buddy about it. Personally I'd be pretty dissapointed if I loaned out speakers and that happened. I think if you go to an electronics store and look at the speakers you'll find that dust caps are somehow irresistable to be pressed in. I have been guilty of it myself in my younger days... much younger days.