First, is the digital coax connection you are talking about a single jack color coded orange on both the TV and the RCA receiver and you are using a single cable with an RCA plug on each end?
There are a few possibilities and you haven't reported back as to whether you tried any of the previous suggestions.
1. If you have only basic cable the audio is analog and if the TV only offers a digital out you are stuck. Even some of the channels of 'digital' cable still use analog audio.
2. Did you turn off the TV's internal speakers?
3. You said previously that digital coax is only available for the DVD input on the receiver. Is there a menu setting to enable the digital input? It may default to analog.
4. If the TV has a digital out then it probably also allows you to set it to pass Dolby Digital or PCM. If the receiver does not have a built-in DD decoder, set the TV to pass PCM.
5. You imply that there is only one VHF/Cable input and yet you get both cable and OTA channels.
Maybe we just have a terminology mismatch here but to get OTA channels requires an antenna and if there is only one RF input on the TV I don't see how you can get both cable and OTA unless you have some kind of splitter that you didn't mention.