Anyone actually done abx testing with hi res music?

B

Beave

Audioholic Chief
So that's similar, but not the same, as what we at my former workplace would have considered resolution. We measured THD+N vs. frequency and vs. level. But not for an approximation of DAC resolution. For resolution we measured dynamic range and SNR. I don't think of THD+N as being the proper measurement for resolution.
 
Ponzio

Ponzio

Audioholic Samurai
found this very interesting on the subject of the lowly MP3
 
Irvrobinson

Irvrobinson

Audioholic Spartan
found this very interesting on the subject of the lowly MP3
I'm surprised anyone can hear the difference between MP3-128 and lossless on that over-processed pop music. I mostly can't.
 
Irvrobinson

Irvrobinson

Audioholic Spartan
Resolve = the difference between the theoretical bits of data and the actual bits of data between 0dBFS and the noise level.

See:
https://labjack.com/support/datasheets/ue9/appendix-b

Or:
https://www.stereophile.com/content/benchmark-dac3-hgc-da-preamplifier-headphone-amplifier-measurements

EG: (first Google result) Burr-Brown 16-bit DAC PCM3006
THD+N -84dB (should be 90db for 16 bit resolution)
How do you calculate 90db for 16bit resolution? BTW, 1 LSB is about 6db.

Two things are obvious from your post. First, since you requoted my link, you don't read or comprehend what anyone else says. Two, you don't really know what you're talking about.

In recognition of your accomplishments in this thread, you've earned a place on my ignore list.
 
Johnny2Bad

Johnny2Bad

Audioholic Chief
found this very interesting on the subject of the lowly MP3 {snip}
Yes, that is a well known post on YouTube. You should go to the original article where they have the files for you to listen to, just as the woman in the video does.

https://www.npr.org/sections/therecord/2015/06/02/411473508/how-well-can-you-hear-audio-quality

Not that it matters, but note the change in title (how well can you hear ... to audiophool?). It does indicate an introduction of expectation bias, which should not exist in a true blind test. Just sayin'

A better test, with music you are familiar with and actually like, would be to select a song from your stored music, create versions with different lossy compression as well as the lossless original ... if you have a HiRez file that would be even better, you could also create a downsampled 16/44.1 version and throw both into the mix. You will have to name them uniquely, one that is descriptive is OK since you aren't going to see them when you take the self-test.

If you're worried about volume matching, just normalize every track, for pop music try -14dB RMS. You might have to fire up your audio editor for that, Audacity for some of you, others will have their own favourites (I would use Amadeus). It won't matter if normalizing causes a resample of your original. You don't need bit-perfect copies of the original, for the purpose of the test they will be close enough.

If you're going to use Classical Music, you should normalize to -20dB RMS. Not everyone likes the stuff, but just as an FYI there are characteristics of Classical that do bring out differences in lossy compression more than Pop music.

If you want to use more than one song, it's better to create multiple playlists, one for each track. It will make it simpler when it comes time to listen to different versions of the same track.

Then create a playlist with just the versions you've selected, have your player software choose them at random, and play them through your main system, take notes, and see how you do. Don't be afraid to turn it up to a nice listening level, and try to choose a time and setting when you are relaxed. Performance Anxiety (some people have problems with tests, period, regardless of how well they know the material) will give poorer results, so really try to be comfortable if you can.

Maybe set your playlist with a few more tracks than actual formats you've created, so that a few appear more than once.

Don't look at the playlist until you are done testing yourself. Maybe get someone to actually operate your computer so that you can't peek, although with care you can pull it off (change the window size so you can't read the playlist, for example, or tape some paper on the screen, randomize a few times without peeking, and so on).

If I've left out a few details as to how to pull this off, well, everybody's bright around here, you'll figure it out.
 
Last edited:
Johnny2Bad

Johnny2Bad

Audioholic Chief
So that's similar, but not the same, as what we at my former workplace would have considered resolution. We measured THD+N vs. frequency and vs. level. But not for an approximation of DAC resolution. For resolution we measured dynamic range and SNR. I don't think of THD+N as being the proper measurement for resolution.
You might like the article at ESP, not that I'm trying to prove anything, just for your academic interest. Touches upon issues in SNR.

Elliot Sound Products: Noise In Audio Amplifiers

http://sound.whsites.net/noise.htm
 
Johnny2Bad

Johnny2Bad

Audioholic Chief
Some words from Charles Hanson, of Ayre Acoustics, who has died just this week but was responsible for designing Ayre's digital products:
" ...
I've definitely performed countless comparisons, always trying my best to ensure to change only one variable at a time. (Sometimes I've unwittingly changed two, but very rarely.) But here is the kicker - the times I've learned the most is when the change (or lack thereof) is *exactly the opposite of what I would expect*.

I never would have expected to hear the effect of dither at the 24th bit, when the DAC chip itself is only good for 19 or so bits. I never would have expect that multiply 16-bit digital audio with 24-bit coefficients (resulting in word lengths of 40 bits) would in the least sound different than using 16-bit coefficients (resulting in word lengths of 32 bits - especially when the output must be reduced back to 24 bits before conversion to analog. These are just a few examples.

Otherwise you simply go through life thinking that you already know everything there is to know and all of your tests just become exercises in confirmation bias.
..."
 
Bucknekked

Bucknekked

Audioholic Samurai
Some words from Charles Hanson, of Ayre Acoustics, who has died just this week but was responsible for designing Ayre's digital products:
" ...
I've definitely performed countless comparisons, always trying my best to ensure to change only one variable at a time. (Sometimes I've unwittingly changed two, but very rarely.) But here is the kicker - the times I've learned the most is when the change (or lack thereof) is *exactly the opposite of what I would expect*.

I never would have expected to hear the effect of dither at the 24th bit, when the DAC chip itself is only good for 19 or so bits. I never would have expect that multiply 16-bit digital audio with 24-bit coefficients (resulting in word lengths of 40 bits) would in the least sound different than using 16-bit coefficients (resulting in word lengths of 32 bits - especially when the output must be reduced back to 24 bits before conversion to analog. These are just a few examples.

Otherwise you simply go through life thinking that you already know everything there is to know and all of your tests just become exercises in confirmation bias.
..."
I surrender. Reading posts like this hurts my mind. I'm joining irv.
 
B

Beave

Audioholic Chief
Some words from Charles Hanson, of Ayre Acoustics, who has died just this week but was responsible for designing Ayre's digital products:
" ...
I've definitely performed countless comparisons, always trying my best to ensure to change only one variable at a time. (Sometimes I've unwittingly changed two, but very rarely.) But here is the kicker - the times I've learned the most is when the change (or lack thereof) is *exactly the opposite of what I would expect*.

I never would have expected to hear the effect of dither at the 24th bit, when the DAC chip itself is only good for 19 or so bits. I never would have expect that multiply 16-bit digital audio with 24-bit coefficients (resulting in word lengths of 40 bits) would in the least sound different than using 16-bit coefficients (resulting in word lengths of 32 bits - especially when the output must be reduced back to 24 bits before conversion to analog. These are just a few examples.

Otherwise you simply go through life thinking that you already know everything there is to know and all of your tests just become exercises in confirmation bias.
..."
I've corresponded with Mr. Hanson (RIP). He seemed to be a nice person who cared about audio a lot, but he had some really wacky beliefs about what he could hear.

For example, we had relays in the audio outputs in one of our product lines, and he swore that relays audibly degraded sound quality - that he could hear when a relay was in a circuit. We had dozens of listeners unable to hear it, but he claimed he could. When I pressed him about proving it in controlled, blind testing, he gave the usual cop outs about how blind testing is flawed.
 
Bucknekked

Bucknekked

Audioholic Samurai
I've corresponded with Mr. Hanson (RIP). He seemed to be a nice person who cared about audio a lot, but he had some really wacky beliefs about what he could hear.

For example, we had relays in the audio outputs in one of our product lines, and he swore that relays audibly degraded sound quality - that he could hear when a relay was in a circuit. We had dozens of listeners unable to hear it, but he claimed he could. When I pressed him about proving it in controlled, blind testing, he gave the usual cop outs about how blind testing is flawed.
where there are extraordinary claims, extraordinary proof is required. Its just never provided.
I did not know Mr Hanson, but I have heard enough extraordinary claims to last me a lifetime.
 
Johnny2Bad

Johnny2Bad

Audioholic Chief
I've corresponded with Mr. Hanson (RIP). He seemed to be a nice person who cared about audio a lot, but he had some really wacky beliefs about what he could hear.

For example, we had relays in the audio outputs in one of our product lines, and he swore that relays audibly degraded sound quality - that he could hear when a relay was in a circuit. We had dozens of listeners unable to hear it, but he claimed he could. When I pressed him about proving it in controlled, blind testing, he gave the usual cop outs about how blind testing is flawed.
His "really wacky beliefs" resulted in some of the best digital products ever made, and he was the design Engineer, not a sales staffer or money man. He must be doing something right and know something about Sound Quality to pull that kind of act off. Yes, he was a polarizing figure, but at least he said what he believed, and stood by his word. Not that common in this Industry.

I have a buddy who is colour blind. When I see Red, Green and Yellow traffic lights, he sees Yellow and two pale pastels that are not quite Black + White but not quite colour either. People like him view traffic lights based on the position of the light (which are standardized), not the colour. Should I insist that he can see Red and Green just because I can?

On the other hand, I have excellent vision; my career depended on it, and at 60, I don't need longer arms to read the fine text on the vitamin bottle. Because I have other health issues, I must take an annual physical to maintain my drivers' license. That includes a vision test ... last exam my left eye was 20:20 and right was 20:15 which means I can see as clearly at 20 feet what most people need to be five feet closer to the chart to see. Should someone who can't see quite as clearly in that eye insist I cannot?

If someone on the internet challenged me to "prove it", I'd go tell them to pound sand; I have better things to do. Maybe Charles did too.

We all have differences in sensory acuity. I have little doubt Charles had excellent hearing; his products demonstrate that if nothing else.
 
Last edited:
B

Beave

Audioholic Chief
"Best digital products ever made" according to whom? I and others I worked with found them outrageously overpriced and no better sounding than run-of-the-mill mass produced electronics (which generally sound quite good themselves).

Really, for the last 20+ years, it's pretty easy to design digital audio products. DAC IC manufacturers supply reference designs and circuit board layout guidelines. Follow the directions and you get a great sounding DAC.

The only things special about his designs were A) the cost B) his claims.

Yeah, yeah, I know, the high end audio press gushed over his designs, and lots of people (who don't do or understand controlled listening tests) probably lavished praise on his products too.

But all that proves is how easily fooled people can be with audio products.
 
Irvrobinson

Irvrobinson

Audioholic Spartan
I'm really surprised at Benchmark Media for this sort of marketecture double-speak, especially that inter-sample overs discussion. To my knowledge, there's no such thing in digital audio as an inter-sample over. In fact, if there was such a thing, I'm wondering how PCM digital audio would work at all. That improperly-designed DAC circuits can clip, and digital clipping is catastrophic, is true, and that recording and mastering can induce digital clipping is true, but to extend the notion to some discussion that Nyquist-Shannon theory is wrong about two samples being insufficient to plot a waveform - which is what the blog post implies - seems really silly. Unless I'm misinterpreting the blog post.
I was browsing the Benchmark website yesterday, and they've augmented the original article on inter-sample overs with some measurements and better examples of the phenomenon. I still get confused by the drawings of sine waves depicting samples on multiple points of a waveform, but it provides compelling evidence that inter-sample overs are real and their effects on DAC performance. The underlying problem still seems to be recordings that have digital clipping in them, but it is interesting that DAC design can mitigate or eliminate the problem.

 
P

PENG

Audioholic Slumlord
I was browsing the Benchmark website yesterday, and they've augmented the original article on inter-sample overs with some measurements and better examples of the phenomenon. I still get confused by the drawings of sine waves depicting samples on multiple points of a waveform, but it provides compelling evidence that inter-sample overs are real and their effects on DAC performance. The underlying problem still seems to be recordings that have digital clipping in them, but it is interesting that DAC design can mitigate or eliminate the problem.

Thanks a lot that is going to take away some of my time for enjoying my music and movie collections this weekend, may be next.. I want to read it and see if I am convinced those are still marketing talks or real, at least potentially.
 

Latest posts

newsletter

  • RBHsound.com
  • BlueJeansCable.com
  • SVS Sound Subwoofers
  • Experience the Martin Logan Montis
Top