When the speakers signature matches, panning sounds have the same sound and volume as they move past you.
If the speakers arent a good match, the sound and volume (differing with frequency) will change.
You most likely are missing out on something, but it depends how well what you have matches.