The ps3 can decode only certain dts audios if it is coming from an .mp4 format, however even in this format the ps3 will not be able to decode all the audio. Also, most of the decode of audio is done via ps3mediaserver, and when I'm decoding dts via ps3mediaserver it gets transcoded into a lossy format of ac3.
When streaming a .mkv w/ dts without having my ps3 doing the transcoding to a lossy format, then the sound will either not play at all or only certain parts of the audio track will play. To get around this little problem, you need a receiver that can do the DTS decoding for you via optical.