Potentially interesting DBT experiment

[Rip Van Woofer]

From The Boston Audio Society:

"After hearing all the claims for better sound with 24 bit/96 kHz
high-resolution audio, the BAS has decided to put them to the definitive
test: a double blind comparison of the best two channel SACD and DVD
Audio recordings we could find, with the same material passed through a
16 bit 44.1 kHz "bottleneck". Can you hear the difference?

"We will be using the Pioneer 563A DVD-A/SACD universal player which
keeps the DSD output of the disc in DSD form without converting to PCM.
The conversion to 16 bit and back will be by a Sony DTC-790 DAT machine
(1996 consumer model) which has been carefully checked to represent the
best in conventional technology."​

I don't belong but evidently got on the list due to joining a local audio club. If we have a BAS member here could they post a digest of the results?

The BAS membership includes leading lights in the audio community from acacdeme and the "biz" alike so any tests they do are likely to be to a high scientific standard.

Also interesting that they're using many Audioholics' favorite budget "universal" player!

 

[WmAx]

From The Boston Audio Society:

The BAS membership includes leading lights in the audio community from acacdeme and the "biz" alike so any tests they do are likely to be to a high scientific standard.

Also interesting that they're using many Audioholics' favorite budget "universal" player!​

But really, it's not probable that the test will be performed to 'high scientific standard'. Such requires significant investment of money, time and resources. The best to hope for is a reasonably well thought out test(demonstration) that is at least far more valuable then tests that do not involve any controls. If positive results are obtained, I would expect extensive measurements to confirm operational parameters that were below known human thresholds of audibility as well as very detailed report of the testing procedure in every aspect. I wonder why they used a DAT machine... a line delay set to mininum delay would be alot easier to implment(synch very close to source). But such testing would be better presented using a computer system featuring a suitble sound card. Eaiser to establish controls as well as having easy to use DBT software available. This allows for the subject to pick the sections of a song and repeat at will. This increases sensitivity to differences.

-Chris​

 

[Rip Van Woofer]

But really, it's not probable that the test will be performed to 'high scientific standard'. Such requires significant investment of money, time and resources (...)

-Chris

OK, OK wise guy. So maybe we can just say it won't be too sloppy!

Actually (maybe this is my scientific ignorance talking - as I often point out I was a theatre major!) I had thought that doing an audio DBT aka A/B/X test was on the low end of complexity and uncertainty as such things go; certainly not as gnarly as, say, drug testing. Just make sure the levels are within 0.01dB, do a statistically significant number of trials (16 to 20 is the number I've heard from various sources), randomize 'x', observe standard double-blind protocol to make sure experimenter bias is eliminated (randomizing 'x' is a big part of that, yes?), agree on the length and nature of the music sample, keep everything the same except for the DUTs (devices under test) and have at it.

Real science you can do at home with a bit of effort...so I thought!

Yeah, I know it's trickier for things like speakers (location cues and such) or CDPs (synchronization).

Always willing to learn.

Anyway, it'll be interesting.

 

[WmAx]

Unfortunately, your description is a vast oversimplification of what is required to produce (1) a highly senstive test [and] (2) does not consider what steps are required to confirm a positive result is valid if one is obtained.

I'll outline some of the basic issues(not going into detail here and I probably missed a couple of things in this summary) that should be addressed in a scientifially rigorous test comparing things that should sound identical according to pre-existing perceptual research:

Sensitivity

It is important to increase listener senstivity as much as possible.

-Listeners should be experienced in sound comparison(s) as opposed to picking average people unless the objective is to test average people that lack the learned skills of sonic comparison.

-Listeners should be in an isolated area/room without peers(or researchers) staring at them -- this may produce possible unwarranted pressure.

-Listener must have access to states being compared beforehand in order to listen casually and become accustomed with differences(if any) in sound.

-Test atmosphere should be comfortable physically(climate, seating, etc.) and have a relaxing atmosphere that should include relaxing color tones and lighting.

-Audio samples should be short and switchable/repeatable instantly upon test subject's will. Using a continous playing signal reduces sensitivey significantly.

Verification(statistically significant(this is an article in itself) positive result obtained in a comparions) Note that these verification steps are not needed if no statistically significant postive results were obtained.

-The intended variable must be meausred extensively to make sure that no extraneous variables produced the positive result such as a distortion caused by a fault connection/contact or an audible aliasing artifact in a poor(defective) DAC, frequency rsponse anomoly, etc..

-The DBT prototocol must be examined to ensure that no extraneous cues allowed identification of the different test states. For example, if a relay box was used and is in proximity of listener, it willl produce audible sounds as switching done. The sounds should be identical regardless of what state is enaged. This must be verified.

-Finally, if all above is verified and a postive result is obtained; this must be peer reviewed and reproduced by additional(ideally seperate party) testing to ensure repeatability.

-Chris

OK, OK wise guy. So maybe we can just say it won't be too sloppy!

Actually (maybe this is my scientific ignorance talking - as I often point out I was a theatre major!) I had thought that doing an audio DBT aka A/B/X test was on the low end of complexity and uncertainty as such things go; certainly not as gnarly as, say, drug testing. Just make sure the levels are within 0.01dB, do a statistically significant number of trials (16 to 20 is the number I've heard from various sources), randomize 'x', observe standard double-blind protocol to make sure experimenter bias is eliminated (randomizing 'x' is a big part of that, yes?), agree on the length and nature of the music sample, keep everything the same except for the DUTs (devices under test) and have at it.

Real science you can do at home with a bit of effort...so I thought!

Yeah, I know it's trickier for things like speakers (location cues and such) or CDPs (synchronization).

Always willing to learn.

Anyway, it'll be interesting.

 

[mtrycrafts]

From The Boston Audio Society:

"After hearing all the claims for better sound with 24 bit/96 kHz
high-resolution audio, the BAS has decided to put them to the definitive
test: a double blind comparison of the best two channel SACD and DVD
Audio recordings we could find, with the same material passed through a
16 bit 44.1 kHz "bottleneck". Can you hear the difference?

"We will be using the Pioneer 563A DVD-A/SACD universal player which
keeps the DSD output of the disc in DSD form without converting to PCM.
The conversion to 16 bit and back will be by a Sony DTC-790 DAT machine
(1996 consumer model) which has been carefully checked to represent the
best in conventional technology."​

I don't belong but evidently got on the list due to joining a local audio club. If we have a BAS member here could they post a digest of the results?

The BAS membership includes leading lights in the audio community from acacdeme and the "biz" alike so any tests they do are likely to be to a high scientific standard.

Also interesting that they're using many Audioholics' favorite budget "universal" player!

I used to belong for a while, no more
They may be the oldest, continuously running audio club in the US? Yes, they do have a very talented membership indeed.
They have been involved in many DBTs in the past and over the years.

 

[WmAx]

I used to belong for a while, no more
They may be the oldest, continuously running audio club in the US? Yes, they do have a very talented membership indeed.
They have been involved in many DBTs in the past and over the years.

WHy no longer a member? Did they refuse to DBT boomboxes?



-Chris

 

[mtrycrafts]

WHy no longer a member? Did they refuse to DBT boomboxes?
-Chris

Yes. Mine was not of high enough caliber

I had too many memberships and long distance the publication had very little after a while that would sustain my interest.

 

[Rip Van Woofer]

Thanks. Although my summary was simplified, I was familiar with most (not all) of the necessary conditions. Of course, it's the "not all" part that's significant!

Not quite "science you can do at home" after all.

Sidebar: the local audio club (Southeast MI Woofer & Tweeter Marching Society) is doing some sort of speaker comparo (I think) at the next meeting in Nov. I've joined and hope to attend. I'll report if it's interesting. Don't have details but the host sent out an email asking if anyone had a "highly regarded speaker" (electromechanical, not human) to bring to the meeting for the program. Mind you, this is a group that according to its Website has "[an] interest in audio...at the highest levels, but our interest in organization is minimal."

 

[mtrycrafts]

Thanks. Although my summary was simplified, I was familiar with most (not all) of the necessary conditions. Of course, it's the "not all" part that's significant!

Not quite "science you can do at home" after all.

Sidebar: the local audio club (Southeast MI Woofer & Tweeter Marching Society) is doing some sort of speaker comparo (I think) at the next meeting in Nov. I've joined and hope to attend. I'll report if it's interesting. Don't have details but the host sent out an email asking if anyone had a "highly regarded speaker" (electromechanical, not human) to bring to the meeting for the program. Mind you, this is a group that according to its Website has "[an] interest in audio...at the highest levels, but our interest in organization is minimal."

That's a great club as well. You might run into Nousaine and other greats there, from time to time. You might even get to listen to his subs at home