Its possible you have them wired out of phase, so when they are playing they are canceling each other, when you stop one cone the cancellation is reduced.
huh, you're on to something here!
I tried unplugging one of the speakers and it had a similar effect.. not quite as clean as when I put pressure on one, but still much improved.
The amp has two ports for speakers, and I'm realizing now that I'm not really sure what the proper set up is, but having both speakers on the same terminals has resolved my issue.
I've attached an image of the LCH and RCH speaker ports, maybe someone can explain them better to me.
Interestingly enough the speakers worked as well as they do in the pictured orientation, while they were wired to the middle two ports (which appear to both be negative, so I'm not really sure whats supposed to be where anymore)
Anyway, they're working now.. and if anyone wants to explain to me why they didn't work while using both 'channels' that would be cool.