I have experimented quite a bit with this and I have B&W 802 speakers as my mains and an HTM1 as a Center channel. I really wish I could get away from the Center, because it is so darn large, that I can't buy a piece of furniture that I wanted to use to dress up my family room.
When I turn my center off and have the mains try to reproduce the voice, there is a loss if intelligibility with the dialogue. The speakers image very well and as long as you don't sit too far off axis, the dialogue is fairly centered, however, the voices are lost by the midrange bass of the speakers.
The difference is honestly night and day. I really wish it wasn't, but a Phantom Center just isn't the same as having a good dedicated center channel.