Human Hearing Acuity Shown to be More Accurate Than Standard Linear Models

[PENG]

A blind test can be just as flawed or MORE flawed than a properly controlled sighted test. Most companies NEVER do DBT's, they at best do blind tests. Many of whom use their own panel of trained listeners in these tests introducing a huge bias they conveniently don't disclose. At that point, you're dealing with a blind test with induced familiarity bias.

In addition, quick ABX testing between speakers often yields preferences towards the brighter/bassier speaker for the inexperienced listener.

Some suggested reading:

The Insanity of Marketing Disguised as Loudspeaker Science | Audioholics

Revealing Flaws in the Loudspeaker Demo & Double Blind Test | Audioholics

How to Skew a Blind Listening Test | Audioholics

Do they really do such test with a group of people instead of having the individuals in isolation like hearing tests? I think in isolation will be better, to avoid the groupthink issue.

I still wonder who would be brave enough to conduct truly unbiased and properly controlled credible blind tests and then even braver to publish it. Please don't ask me why I say this..

 

[gene]

Do they really do such test with a group of people instead of having the individuals in isolation like hearing tests? I think in isolation will be better, to avoid the groupthink issue.

I still wonder who would be brave enough to conduct truly unbiased and properly controlled credible blind tests and then even braver to publish it. Please don't ask me why I say this..

Yes and some companies have their staff go so far as to write an article/review of their speaker compared to a more prestigious brand and then publish it on other AV websites as if it were an independent review.

We have done two blind tests with a listening panel unfamiliar with the speakers. However in our 2nd test, Clint started spotting the Klipsch speaker b/c it was the briggest and bassiest of the bunch. After about 1hr of switching back and forth, he figured it out. Experienced listeners know what to listen for, untrained ones typically don't.

 

[3db]

I don't see the value in a BLT for speakers as speakers are at best a subjective choice based on ones' ears. What what person likes, the other may despise/loathe. Solid state amps and AVRS, are a different ball game. Pushing aside the features for the moment, I would really like a BLT test done on AVRSs and AMPs where the same speakers are used in the same acoustic environment and that the speakers are capable of being easily driven by the weakest candidate at some pre determined volume level. I realize more difficult loads would began to show up audably (if that's a word) if the weakest contender gets pushed beyond its design limits. Buying a more powerful amp allows the purchase more choice of speaker and room size and as a result sound better than a less powerful amp but that's due to the lesser amp being pushed out of its intended design power delivery limits.

 

[mtrycrafts]

I don't see the value in a BLT for speakers as speakers are at best a subjective choice based on ones' ears. What what person likes, the other may despise/loathe. ....

Well, but, always a but, or two
Research done up at NRC shows that most do like speakers that have very good FR spec as close to flat as they can make it. I am pretty sure you have seen the graphs Sean Olive has on his web site showing this and what happens to perception when blinded.

While subjectivity is part of it, seeing a speaker can and will alter your sonic perception just as testing any other audio gear sighted. The eye brain interaction is just too powerful

 

[gene]

Well, but, always a but, or two
Research done up at NRC shows that most do like speakers that have very good FR spec as close to flat as they can make it. I am pretty sure you have seen the graphs Sean Olive has on his web site showing this and what happens to perception when blinded.

While subjectivity is part of it, seeing a speaker can and will alter your sonic perception just as testing any other audio gear sighted. The eye brain interaction is just too powerful

Realize too that the NRC research is 30 years old and the measurements taken on loudspeakers is a small snapshot of actual performance. I don't buy all the gospel that manufacturers like to attach to the research. I've had many NRC favored speakers in for review that didn't fair too well in long term listening tests to some of their brethren. I've also had the Sean Olive favorite Infinity P363 speakers in for a review and long term listening test and they weren't nearly as good as the Harman marketing slides lead their dealers to believe With our panel of experienced listeners, they got quite fatigued by the bright treble they produced compared to tamer more neutral sounding speakers. I would NEVER recommend the P363s at their retail price but at $200/pair they're a great deal though the Andrew Jones towers are a compelling alternative.

 

[3db]

Realize too that the NRC research is 30 years old and the measurements taken on loudspeakers is a small snapshot of actual performance. I don't buy all the gospel that manufacturers like to attach to the research. I've had many NRC favored speakers in for review that didn't fair too well in long term listening tests to some of their brethren. I've also had the Sean Olive favorite Infinity P363 speakers in for a review and long term listening test and they weren't nearly as good as the Harman marketing slides lead their dealers to believe With our panel of experienced listeners, they got quite fatigued by the bright treble they produced compared to tamer more neutral sounding speakers. I would NEVER recommend the P363s at their retail price but at $200/pair they're a great deal though the Andrew Jones towers are a compelling alternative.

Speaker manufacturers use the NRC's facilities to conduct measurements and not to abide by what ever design principles they came up with even though the fundamentals are still sound to this day. I know PSB uses the NRC facilities extensively for measurement but they also include listener panels as well to fine tune their designs. Long term listening tests are still subjective and listener dependent and the end results that your listening panel has experienced aren't exclusive to designs which employed the use of the NRC facilities. Its up to the manufacturer of the speakers on how the end results sounds, not the NRC.

 

[gene]

Speaker manufacturers use the NRC's facilities to conduct measurements and not to abide by what ever design principles they came up with even though the fundamentals are still sound to this day. I know PSB uses the NRC facilities extensively for measurement but they also include listener panels as well to fine tune their designs. Long term listening tests are still subjective and listener dependent and the end results that your listening panel has experienced aren't exclusive to designs which employed the use of the NRC facilities. Its up to the manufacturer of the speakers on how the end results sounds, not the NRC.

I agree but my point was the fact that some companies rely solely on the limited scope of measurements standardized by the NRC to govern product design. The distortion testing is very limited in this regard and can lead to false conclusions. A good designer also does real listening and final voicing of a speaker in a real room for the final stages of design.

 

[3db]

I agree but my point was the fact that some companies rely solely on the limited scope of measurements standardized by the NRC to govern product design. The distortion testing is very limited in this regard and can lead to false conclusions. A good designer also does real listening and final voicing of a speaker in a real room for the final stages of design.

I agree that some companies don't do it right and you can hear that in their designs.

 

[GO-NAD!]

I don't see the value in a BLT...

You take that back! BLT's are delicious!

 

[PENG]

Long term listening tests are still subjective and listener dependent and the end results that your listening panel has experienced aren't exclusive to designs which employed the use of the NRC facilities. Its up to the manufacturer of the speakers on how the end results sounds, not the NRC.

Agree, and I still think measurements are the lesser of the evils. At this stage and age, if measurements are done with the proper equipment and the relevant things are measured, what measure well should sound well. Again, the keys are the whats and hows, if the NRC are not doing all the right thing then it is about time they do, or we need someone else to do it. I will not go by someone else ears and brains to tell me what sounds good in their listening environments.

 

[Ampdog]

With regard to Tom Andry's article "How to screw a Blind Listening Test", the argument to me is not so much whether it is posible, but whether that should necessarily form a disqualification, as some 'opponents' seem to want to claim. (Indeed everything can be 'screwed'!) There seems to be nothing which cannot be taken care of.

Regarding the use of such for loudspeaker testing there is another factor that I have not noticed mention of. It is the influence on the room acoustics of all those other passive loudspeakers around waiting their turns to be used, acting as absorbers or accentuators of certain frequencies! I recall at least one test where such influences were measured. It is patently impractical to move test loudspeakers in one pair at the time, though shorting the input terminals of passive ones were shown to have had measurable if small effect to curb such influences. (Such effects would naturally be dependant on the resonances of passives, et al.) At its simplest, one can use an oscilloscope to watch all the passive units merrily microphoning along.

Indeed matters are not simple!

 

[3db]

if the NRC are not doing all the right thing then it is about time they do, or we need someone else to do it. I will not go by someone else ears and brains to tell me what sounds good in their listening environments.

The NRC supplies an anoechic chamber for measuring purposes and I'm not even sure they supply the measuring equipment. An anoechic chamber is hardly what I consider a listening environment. I'm not sure what you are driving at.

 

[gene]

Agree, and I still think measurements are the lesser of the evils. At this stage and age, if measurements are done with the proper equipment and the relevant things are measured, what measure well should sound well. Again, the keys are the whats and hows, if the NRC are not doing all the right thing then it is about time they do, or we need someone else to do it. I will not go by someone else ears and brains to tell me what sounds good in their listening environments.

The NRC isn't the problem. It's some of the manufacturers that try to paint too broad a brush with the results they've gathered nearly 3 decades ago.

The problem is no matter how much a manufacturer would try to convince you, sound quality is still very subjective. Not everything we can measure matters and not everything that matters can accurately be measured or quantified.

 

[mtrycrafts]

The NRC supplies an anoechic chamber for measuring purposes and I'm not even sure they supply the measuring equipment. An anoechic chamber is hardly what I consider a listening environment. I'm not sure what you are driving at.

Toole and Olive were employed by them for a long time and did actual research there when he published his JAES papers and into the 1990s.

 

[PENG]

The NRC isn't the problem. It's some of the manufacturers that try to paint too broad a brush with the results they've gathered nearly 3 decades ago.

The problem is no matter how much a manufacturer would try to convince you, sound quality is still very subjective. Not everything we can measure matters and not everything that matters can accurately be measured or quantified.

I can see your point about sound quality being subjective and not everything we can measure matters. However, for me anyway, if we can measure the sound quality of unamplified live concerts and compare that with a home system using the best equipment available including spectrum analyzers, then that is something I will value more than opinions and impressions of professional reviewers or other forum members. As a frequent classical concert goer, I do also understand that people may actually prefer certain kind of sound quality of reproduced music to live music. Again I am aware of the issue of subjectivity and would like to avoid making my choice of equipment base on subjective evaluations (including my own).

 

[PENG]

The NRC supplies an anoechic chamber for measuring purposes and I'm not even sure they supply the measuring equipment. An anoechic chamber is hardly what I consider a listening environment. I'm not sure what you are driving at.

I wasn't referring the NRC at all but the listening environment of reviewers including forum members such as you and I who regularly report/offer their own listening experience. Those (including mine) are subjective opinions and impressions that I do not value much. I value much more measured performances shown by all sorts of data and graphs, but would like to have ways to know that such measurements and plots are credible with full disclosures of the details and specs of the instruments used and the credentials of the reviewers as well as those who took the measurements. For example, I would like to know the qualifications and experience of Mr. Atkinson for sure. When I read the Stereophile reviews, I only focus of their description of the products features, specs, and then I skip through the fluff words and jump right to JA's section. That's just me, YMMV obviously.

 

[3db]

Toole and Olive were employed by them for a long time and did actual research there when he published his JAES papers and into the 1990s.

Like I stated previously, the NRC (thanks to you I know now that it was in conjuction with Toole and Olive) lay the ground work and the principals they came up with are fundamental guide lines that speaker manufactures follow. Like Gene pointed out, NRC may need updating in the distortion testing but everything else seems to be accurate. I think it would be safe to say that the one update required doesn't preclude that the other measurements as being wrong or not applicable. There are far too many good sound speakers out there that employ the ground work of the NRC in conjuction with Toole and Olive. However, this does not preclude the fact that the responsbility of how a speaker sounds rests with the manufacturer and not the NRC. Fine tuning is required. PSB does it and I'm sure there are others as well.

 

[3db]

The NRC isn't the problem. It's some of the manufacturers that try to paint too broad a brush with the results they've gathered nearly 3 decades ago.

The problem is no matter how much a manufacturer would try to convince you, sound quality is still very subjective. Not everything we can measure matters and not everything that matters can accurately be measured or quantified.

I disagree that everything we measure in a loudspeakers matters. I think everything we can measure about a loud speaker matters as its the basis of a good performing loudspeaker. Look at Bose who does not employ the research done by Tool and Olive at the NRC. and you can hear it immediately... " boom and sizzle"

I do agree with the part that not everything that matters is accurately measured. We can't measure subjectiveness and nor one can design around subjectiveness that a speaker will be loved by everyone.

 

[Goliath]

A blind test can be just as flawed or MORE flawed than a properly controlled sighted test. Most companies NEVER do DBT's, they at best do blind tests.

"Properly controlled sighted test" is a misnomer. If it is sighted then by definition it is uncontrolled. Listener bias is so prevalent in a sighted test that it is almost exclusively relied upon, and yet people draw their conclusions from such egregiously flawed practices.

Of course, most audio companies selling high-end amplifiers, DACs and cable would never perform DBT's of their products because it would result in a monumental loss of revenue.

Many of whom use their own panel of trained listeners in these tests introducing a huge bias they conveniently don't disclose. At that point, you're dealing with a blind test with induced familiarity bias.

Double-blind testing would be preferable over blind testing. What the Sean Olive testing confirms is that whether trained or untrained, people seemed to prefer the flattest sounding speaker in a double blind test. So as far as preference goes, in their testing, the degree of formal training did not seem to have any abnormal impact on the results.

However with double blind results of amplifiers, cables, dacs, power cords, etc - you would think that some degree of listener training would be included as part of the test protocols. But it almost doesn't matter - when after 30-40 years, the results are just terribly predictable at this point.

So you could argue that all those participants where deaf, or the test protocols where all fudged, or some other conspiracy theory - or you could reach the conclusion that the results are accurate and true and indicative of observable reality.

 

[Goliath]

In addition, we have close to a century worth of psycho-acoustic research that lends credence to all the double-blind research ever conducted. I think at this point if people still feel compelled to argue with the testing, then I think denial is a strong factor that cannot be ignored.

After all, if your long-held cherished beliefs were revealed to be nothing more than the result of an improperly set up comparison, where levels were mismatched, components were slow-switched and sighted, where every form of bias imaginable was influencing your results, you would either carry on believing, cognitive dissonance-style, or adapt and evolve.

How many people adapt and evolve?