I know I already state my opinion... but I wanted to clarify what I did when I had this same question.
My setup:
Klipsch RB-75s, RC-7, (4) RS-7s.
SVS PC+ 20-39
Emotiva DMC-1 and MPS-1 doing the work.
Source... some Loseless (ALAC) through my Sonos, output TosLink.
Music was just some Top 40 Pop type stuff... nothing critically acclaimed for testing or anything. Just music I know and enjoy.
I played around with all the settings in the DMC-1 changing all the speakers to large and small and then just doing the mains and center as large. As large I noticed more bass response at normal volume levels, maybe a more "complete" sound... but as soon as I turned up the volume to anything enjoyable... things washed out and got muddy right away. I don't feel I have an overly critical ear... but I could tell a big difference between trying to send those frequencies to speakers vs. letting the Sub take care of it.
Did I know they should all be set to small before I started? Yes...
Did I know that the particular speakers I have are HT speakers and NOT intended to provide decent low frequency response? Yes...
But... I just wanted to see for myself what the differences would sound like and if my ear preferred one setting over the other. Surprisingly with my setup and my sub... I definitely preferred everything as small and when listening at obnoxious volume levels... no distortion and no perceived stress.
Rayo