Flushing Out Myths in Consumer Audio

[krabapple]

We sure are grateful to have you around, Krabapple. What would we do without you? Hmmm?

Gee, are you following me around the forum?

As usual, you have misrepresented what I said. (I realize some concepts are difficult to comprehend, but this sounds like merely more of your impudent ego at work...just spoiling for a fight.)

My point was...clear English now...that these two disparate concepts have been argued on different threads in this forum...as though they are the same thing...which of course they are not.

Gee, who sounds pugnacious here?

It's you who've proposed that these are two real and distinct mental operations. That when the audiophile buys his new amp and plugs it in and hears that 'open soundstage' and 'more detail', it's not an identification, it's not a task involving memory, it's just an...experience? Because , I suppose, tjere's no 'pressure' to 'pick'? OK, I'll humor you.

Nothing is stopping DBT skeptics from acclimating themselves for as long as they like, before doing the comparison blind. Do whatever you imagine is necessary to make yourself as discriminating as you can. Bathe yourself in the 'audio experience' as long as you deem necessary. The only requirements after that are 1) that during the actual test, the only 'clue' you have for judging the sound, is the sound and 2) you do enough trials to make for some some decent statistics.

Alternately, if you insist that that's no darn good, that the *mere act* of consciously attempting to tell one amp from another throws the game, how do you propose to get around the *known, demonstrable, scientifically uncontroversial effects* of sighted bias, which are surely operating during both your 'audio experience' and your 'identification', hmmm?

Even if you wuss out, deny the scientific method, and insist that there just is no way to verify the real sonic basis of a subjective audible difference, shouldn't you at least, *always*, acknowledge the likelihood of your 'differences' being completely imaginary, unders such conditions? It's what I do, btw, because of course I *don't* do DBTs on everything I buy. Unless there is a sound (and generally trivial) reason for expecting things to sound different -- e.g. one AVR has DSP engaged, one doesn't; one amp's tubed, the other's not, etc -- then I am perfectly happy to admit that any differences I thought I heard in a sighted, unlevelmatched comparison, could be imaginary or due to trivial level mismatch (e.g. one amp's '11' not the same as the other's) -- not 'intrinsic' to the gear. It's called acknowledging the psychological facts of life.

Funny how few audiophiles ever have the brass to do that.

 

[highfihoney]

You misunderstand. But tell me, why *shouldn't* two amps match at all three frequencies, when they are matched at one? Doesn't that mean at least one of the amps is acting as an equalizer? Is that what amps are designed to be? Or are they designed to be more or less 'flat' from 20-20kHz?



Again, how is it 'rigging' to test whether one amp is potentially 'broken' ?

.

Sorry but i did'nt misunderstand,if both amps match in all frequencies right from the factory then fine & dandy its a legit test but once ANY alteration to any signal or any other level of either amplifier is made the test has become invalid,in order for any comparison test to be valid NOTHING can be adjusted,altered or changed in any way then an instant AB test can be performed.

You ask how is it rigging to test if one amp is broken,the only reason to match any levels is not to test if an amplifier is broken but rather to find out what adjustments need to be made to confuse people into thinking they are not hearing different amps,what happens once each amp is returned to its natural state?

 

[MDS]

You have two amps that you want to test using DBT protocol:

- You listen to each (sighted) for as long as you want using as many different types of music that you want.
- After said listening experience you have formed an opinion as to which amp sounds 'better'.
- Now you take away the knowledge (from sight) as to which amp is actually playing. If you did indeed form a strong opinion of which amp sounded better then you should be able to pick out which of the two amps under test is playing using only your ears, but to be statistically significant you have to pick the right one more times than would occur by pure chance alone (obviously by pure chance you will be right roughly 50% of the time because after all there are only 2 amps under test).

Pretty simple and straightforward, no?

 

[highfihoney]

You have two amps that you want to test using DBT protocol:

- You listen to each (sighted) for as long as you want using as many different types of music that you want.
- After said listening experience you have formed an opinion as to which amp sounds 'better'.
- Now you take away the knowledge (from sight) as to which amp is actually playing. If you did indeed form a strong opinion of which amp sounded better then you should be able to pick out which of the two amps under test is playing using only your ears, but to be statistically significant you have to pick the right one more times than would occur by pure chance alone (obviously by pure chance you will be right roughly 50% of the time because after all there are only 2 amps under test).

Pretty simple and straightforward, no?

Hi mds,to me its not that simple,why must an individual pick anything? nobody needs to see any of the gear in question,i'll never understand whats wrong with simply switching back & forth & noting the differences if any that are heard.

Example,all gear is not seen,amp 1 is playing allowing time to take a few notes before switching amps.

amp 1, notations,amp one has too much bass & the highs sound harsh.

amp 2, notations,amp 2 has weaker bass & the highs are not harsh.

There is no bias from this type of AB testing & no memory skills are required, To me its not nor has it ever been about picking which amp sounds better rather to hear that there are differences in the way different amps sound.

Either way i still have fun messing with new gear

 

[highfihoney]

By this logic, magazine reviews as written are equally useless. So are auditions in store. For that matter, so are auditions at home, unless you can 'replicate' the conditions every time. Do you agree?

Do you think that consumers should ignore results of medical trials, too? After all, they're not likely to replicate those at home either.

Sorry krabapple,my fat fingers screwed up my post that you responded too.

my post should of read like this.

It is impossible to set up a fair dbt where the conditions are NOT able to be replicated constantly by the consumer(end of quote from my post)meaning that most consumers use the amp as a plug & play device & all levels matched does not apply because they dont have the ability to match all levels let alone keep all levels at a constant under different power requirements while listening.

 

[Tomorrow]

Gee, are you following me around the forum?



Gee, who sounds pugnacious here?

It's you who've proposed that these are two real and distinct mental operations. That when the audiophile buys his new amp and plugs it in and hears that 'open soundstage' and 'more detail', it's not an identification, it's not a task involving memory, it's just an...experience? Because , I suppose, tjere's no 'pressure' to 'pick'? OK, I'll humor you.

Nothing is stopping DBT skeptics from acclimating themselves for as long as they like, before doing the comparison blind. Do whatever you imagine is necessary to make yourself as discriminating as you can. Bathe yourself in the 'audio experience' as long as you deem necessary. The only requirements after that are 1) that during the actual test, the only 'clue' you have for judging the sound, is the sound and 2) you do enough trials to make for some some decent statistics.

Alternately, if you insist that that's no darn good, that the *mere act* of consciously attempting to tell one amp from another throws the game, how do you propose to get around the *known, demonstrable, scientifically uncontroversial effects* of sighted bias, which are surely operating during both your 'audio experience' and your 'identification', hmmm?

Even if you wuss out, deny the scientific method, and insist that there just is no way to verify the real sonic basis of a subjective audible difference, shouldn't you at least, *always*, acknowledge the likelihood of your 'differences' being completely imaginary, unders such conditions? It's what I do, btw, because of course I *don't* do DBTs on everything I buy. Unless there is a sound (and generally trivial) reason for expecting things to sound different -- e.g. one AVR has DSP engaged, one doesn't; one amp's tubed, the other's not, etc -- then I am perfectly happy to admit that any differences I thought I heard in a sighted, unlevelmatched comparison, could be imaginary or due to trivial level mismatch (e.g. one amp's '11' not the same as the other's) -- not 'intrinsic' to the gear. It's called acknowledging the psychological facts of life.

Funny how few audiophiles ever have the brass to do that.

I said all that? Wow. Really?!

 

[Tom Andry]

Please read this article. DBT is not that complicated nor is it the only valid testing methodology. But arguing that level matching is somehow skewing the test is just silly. Volume difference is well documented as a source of bias. Level matching is imperative to a valid test of speakers/amps/etc.

It is impossible to set up a fair dbt where the conditions are NOT able to be replicated constantly by the consumer

You don’t understand the interpretation of a test of this sort. When research is conducted, only the uninitiated and the uninformed believe that anything is proved. It isn’t. What is shown is that at that particular time with that group of subjects in that setting, X results were recorded. The next step is for another researcher to replicate the study with one or more variables changed. Over and over. Each time the same results are reached, the stronger the theory is supported. Eventually, the theory becomes so supported that it is used to formulate other theories. But once even ONE result is reached that contradicts the rest of the data - that must be addressed. Either the theory is modified to fit the new information gathered from that test, or the test is shown to be invalid or a statistical abnormality.

 

[krabapple]

I said all that? Wow. Really?!

No. Hence the word 'if'. (Always glad to help you through the hard parts.)

 

[mulester7]

You have two amps that you want to test using DBT protocol

.....then you wake up and take something for your heartburn that brought on this bad dream.....

 

[krabapple]

You don’t understand the interpretation of a test of this sort. When research is conducted, only the uninitiated and the uninformed believe that anything is proved. It isn’t. What is shown is that at that particular time with that group of subjects in that setting, X results were recorded. The next step is for another researcher to replicate the study with one or more variables changed. Over and over. Each time the same results are reached, the stronger the theory is supported. Eventually, the theory becomes so supported that it is used to formulate other theories. But once even ONE result is reached that contradicts the rest of the data - that must be addressed. Either the theory is modified to fit the new information gathered from that test, or the test is shown to be invalid or a statistical abnormality.

Based on long experince with 'audiophiles', here's the message I predict the average golden ear takes away from all that : "Since someone else's DBT doesn't disprove what *I* hear, I'm gonna keep assuming that what I hear is real, thanks"

In other words, they won't acknowledge the possibility of error...except as it applies to *other people's DBT results*.

 

[mulester7]

Volume difference is well documented as a source of bias.

....and forever may it remain so.....

 

[mtrycrafts]

BTW, ideally speakers would be auditioned blind but it's not as essential as speakers have proven time and again to sound entirely different. One interesting thing that I heard (I've lost the link) is that sighted listening of speakers can introduce bias and, in general, when blinded, people tend to choose speakers with the flattest frequency response (as opposed to the prettiest veneer or coolest curves).

You heard correctly. If sighted amp test is biased, so is sighted speaker comparisons, even if they are audibly different.

Dr. Floyd Toole has a paper or two on this.

This one is about 4 speakers judged by grading a number of parameters, sighted and DBT. Totally different results from DBT. Speaker's bias the listener just as much as anything in audio.
Toole, F. E. and Olive, S. E. ' Hearing is Believing vs Believing is Hearing: Blind vs Sighted Listening Tests and Other Interesting Things,' 97th AES Convention, Nov 1994, Print #3894.

This one is about what people prefer from a speaker. Similarly at this link from Mirage, tells to NRC story where Tool did his research with others for 20+ years. Very large population, 2000 and 330 speakers.
http://miragespeakers.com/nrc_story.shtml

Toole, F. E. Loudspeaker Measurements and Their Relationship to Listening Preferences,' Part 1, Journal of the Audio Engineering Society, Vol 34, No.4, Apr 1986, pg 227-235; Part two, JAES Vol 34, No.5, May 1986, pg 323-348.

 

[mtrycrafts]

using peer-review discussion and publication,

Rest assured, there is no such peer reviewed paper in a journal. So what? None has been done well to draw conclusions?

http://www.sandia.gov/LabNews/LN04-24-98/detector_story.html

Are you telling us that since this was not published in a peer Journal, it has no meaning? That that detector is for real???

Sighted comparisons to draw meaningful conclusions is meaningless, useless.

 

[mtrycrafts]

Toole and Olive of Harman/JBL have gone to great lengths to use DBTs for speaker preference studies , and have produced peer-reviewed p\papers about it, because it's pretty much a given that loudspeakers really sound different. .

I think they used DBT protocol for a number of reasons:
That is the only gold standard to advance science.
Speakers are what you hear and is the most difficult to get right. Amps are nothing. Wires are less than nothing.
If it wasn't for the 'golden ears' the economy would be a whole lot smaller and a whole lot more unemployed
So, it has a place in the economy, sort of. Employs the ...(you fill it in)

 

[mtrycrafts]

People being "steadfast in beliefs," is exactly the source of the problems I see on this site. While the "objectivists" might ultimately be right on many matters, each thread degenerates into broad lectures on audio principles rather than discussion about the actual pieces at hand.

What is there to discuss about that component when evidence is lacking from a poster to support their position? On the other hand, audio and human principals don't need specs to discuss why it would sound the same, or different, for that matter? Posts are put forth from a rather weak position, a biased observation. What more is there to discuss then? It is meaningless to discuss flawed perception without those audio and human principals of know science?



Not only that, but the "objectivists" often come in with a condescending tone armed with nothing but assumptions and principles.

Assumptions? Hardly. The body of evidence goes back 30+ YEARS. not sure what you are trying to assume.


It ends up creating fruitless debate rather than meaningful discourse.

No. It is meaningless to closed minds(yes, those claimed open minds are really closed) and some, who want to know what is in the real audio world and audio science, they will research the cited papers and draw their conclusions. The others, whose belief system is so fragile, well that is what you see.



and, unfortunately, it happens in almost every thread.


Only when testable claims are made anyhow. And why not? Should pseudo science be allowed to flourish unchallenged??? That is why that green pill to improve car gas mileage is racking in $20+ million. Or, the billion $ market for magnet therapy. Am I supposed to just pass on it???


Unless people have actual experience with the equipment at hand (or at least know some key technical specifications), I don't really see the point in interjecting and applying generalizations.

That in itself is a problem. You may place too much weight on personal experience of dubious quality and value. My personal experience is that a bucket of water bends metal pipes and the air straightens them out.

 

[Tomorrow]

Rest assured, there is no such peer reviewed paper in a journal. So what? None has been done well to draw conclusions?

http://www.sandia.gov/LabNews/LN04-24-98/detector_story.html

Are you telling us that since this was not published in a peer Journal, it has no meaning? That that detector is for real???

Sighted comparisons to draw meaningful conclusions is meaningless, useless.

Nope. Not saying anything like that. I'll repeat....some are confusing the issues.

 

[Tomorrow]

No. Hence the word 'if'. (Always glad to help you through the hard parts.)

Does anybody know a cure for a bad case of the Krabs?

 

[mtrycrafts]

Actually, I don't see many DBTs going on in this forum. I do see "subjectivists" saying that they hear differences and don't really want to subject all of their beliefs to DBT. I also see subjectivists arguing against the effectiveness of DBTs. Nevertheless, none of that happens unless there is an attack on their beliefs first. All I'm saying is if you (not meaning you specifically) are going to attack someone's conclusions, do so with specific facts or experiences, rather than generalizations that may or may not be true when applying it to the specific case at hand.

No one is attacked but that is another example of perceptions. Flaws are pointed out, the errors are pointed out. In the end, if one is not interested in learning, knowing facts and reality, nothing will change that.
However, there are many more members or visitors who just read the posts and learn, their curiosity is perked and expand their knowledge on their own.

So, these discussions are not only for the benefit of the posters, but all those silent onlookers. There is a real benefit even if we don't see it at any moment.

 

[Sleestack]

No one is attacked but that is another example of perceptions. Flaws are pointed out, the errors are pointed out. In the end, if one is not interested in learning, knowing facts and reality, nothing will change that.
However, there are many more members or visitors who just read the posts and learn, their curiosity is perked and expand their knowledge on their own.

So, these discussions are not only for the benefit of the posters, but all those silent onlookers. There is a real benefit even if we don't see it at any moment.

Your perception is that no one gets attacked. We simply don't share the same perception there. You are drawing these conclusions based on what? How do you know that a majority of the members and visitors don't get irritated and decide to avoid the thread all together? I'm not saying that is the case, but you make some rather presumptuous statements. I'm just wondering if you simply have an inflated view of the value of these discussions or you have conducted some type of poll.

 

[AdrianMills]

I need a reality check

Mtrycrafts,

have you read those threads that I linked to? Anyone else? The admins and the subjectivists there are insisting that I was continually "rude and insulting". Please, I really need a reality check here by an unbiased observer - was I or are they just caught up in a little confirmation bias over there? Or maybe am I?

Sometimes when you're continually told a thing by a number of people you can start to doubt your own judgment.