If the only thing you're doing is changing channels, your remote's programming may have a command for changing the sound mode. You might also look in the TV's setup- if HDMI control or CEC are on, it screws up things that don't need to be messed with.
Which model of remote? If it's the Ultimate, make sure nobody made any changes to the setup on the remote itself.