Swap them left to right speaker and see if the same thing happens. If it doesn't, then it isn't a problem. If it does, then there could be an issue with the speaker that has that behavior. Does it actually sound different, like something is missing, or does the sound seem normal?