Speaking from my experience, I feel like a dedicated DAC is a close second behind your loudspeakers for most important component in your audio chain.
Even my wife, who couldn't care less about audio quality, still raves about what a difference the XDA-2 made in our system.