Maybe your phone is not at full line output. Maybe your Xbox is set on variable output rather than fixed.
I was wondering that too. That's why I asked him to clarify. I'm pretty sure there's a volume or "gain" adjustment in the Xbox's audio settings. Possibly the phone too.
@stormglove, just how far do you get from your speakers while you're be-bopping around the house doing stuff? Are you talking going into other rooms? Upstairs? Is it loud enough when you're in the room with them?