The Insanity of Marketing Disguised as Science in Loudspeakers

[GO-NAD!]

...The problem with the obejectivists has been the crap rammed down our throats by the subjectivests and their golden ears and megadollar wallets. We grow tired of unsubstantiated claims by the "audio bunny" reviewers who's subjective fluff is taken as gospel....

Absolutely! Objectivists need something tangible and concrete to help judge the quality of a loudspeaker - even if that concrete is actually quicksand in disguise.

It’s important to realize that the audio market could be best described as 90-10% where 10% are audiophiles/hobbyists (like us) and the other 90% are just casual listeners. The 90% would prefer ‘good’ sound over ‘mediocre’ sound if given the choice assuming it even entered their mind in the first place. It usually doesn’t.

Gene, those numbers may be pretty accurate, but I'd go a little bit farther. Of the 10% who are audiophiles/hobbyists, how many do you think can/will go through the agony of bringing home several pairs of tower speakers (I won't even get into full surround systems) and try to do a thorough comparison? Maybe 1%? The rest of us don't have the time or endurance for such an exercise.

I tried it with just 2 pairs of towers (Paradigm Monitor 9 and Polk RTiA7) and it was a pain in the a**. Sure, I could tell them apart - but which ones were better? It was almost a coin toss, but I picked the 9's mainly because of their greater sensitivity and lower power requirement. The Polks went a bit deeper, but with subwoofers, who cares? Now, some folks on this forum wouldn't insult their ears to listen either, but these were available in my area and within my budget. There were certainly other models to choose from, but the effort required to do a thorough comparison of everything available would've been more than I was willing or able to go through. Listen to them in the stores? Sure, I listened to everything in my area, in my budget - probably 6 -7 pairs. But, in different stores with different acoustics and on different days - it was a fools errand and a lesson in frustration.

With your background(Gene), you could probably do a creditable comparison under those circumstances, but for me, it was almost eeny-meeny-miney-mo. I'm willing to bet that a significant number of people - if not the majority - within the 10% are in the same boat as me. Sure, we could tell 2 different speakers apart if they're side-by-side, but unless the difference in quality is glaring, how many of us can honestly and accurately say which one is better? That's why we want measurements! We'd prefer that they are standardized, but apparently we can't have that.

Then, there are internet direct manufacturers. If we should rely on in-home trials as our method of choosing, how many of us are gonna order from several ID dealers and return all but one model? As far as I know, only Aperion pays shipping both ways. So who wants to pay for returning all those speakers? I realize that few people are going to order from more than one ID seller, but I think you see my point? That's why we need to give significant weight to measurements - flawed as they are - when choosing loudspeakers.

 

[AcuDefTechGuy]

I tried it with just 2 pairs of towers (Paradigm Monitor 9 and Polk RTiA7) and it was a pain in the a**. Sure, I could tell them apart - but which ones were better? It was almost a coin toss, but I picked the 9's mainly because of their greater sensitivity and lower power requirement.

Would you be able to tell which one had greater sensitivity and lower power requirement in a DBT?

It was a coin toss. Which would you pick in a DBT?

DBT, measurements, opinions are all good. Keep them coming. But we all know there are so many factors, and every factor is every bit important to us.

Brand and aesthetics are damn important to me.

 

[GO-NAD!]

Would you be able to tell which one had greater sensitivity and lower power requirement in a DBT?

It was a coin toss. Which would you pick in a DBT?

I actually tried a SBT. I got my wife to switch between speakers and randomly vary the volume, so that I couldn't identify either speaker because of different sensitivities.

I guess it's easy for you - just buy the most expensive speakers that AH recommends.

J/K!

 

[avnetguy]

I think it is a good article and really the 4th paragraph in the conclusion section (testing in home environment) IS the test the consumer needs to put, by far, the most weight to IMO.

Steve

 

[AcuDefTechGuy]

I actually tried a SBT. I got my wife to switch between speakers and randomly vary the volume, so that I couldn't identify either speaker because of different sensitivities.

I guess it's easy for you - just buy the most expensive speakers that AH recommends.

J/K!

Did AH recommend my speakers?

It would be nice to me.

I like brand name, aesthetics, compliments, nice words, and all that.

I was going to buy the RBH T30, but I just did not like the aesthetics at all.

 

[gene]

Gene,

Very interesting thread. Gene, here is the deal. I would only really trust Don Keele and his measurements. I have corresponded with lots of people in the industry. He stands out as the most honest and the most technically proficient in speaker measurements. Just my opinion from exchanges with people in the industry. As Atkinson said (and I hate to quote him) "Don wrote the book" for us old guys. The point I guess is that I am a skeptic of you and the rest of the industry. You sell speakers and that is a bias that I cannot just ignore. When I get this "you can't trust measurements" (unless they are mine), and you can't trust blind or DBT tests (and the person writing sells speakers), what would you think or should this be let's show dirty laundry (and who is the most bias)?

In my industry, you cannot even HAVE the APPEARANCE of conflict of interest. You can say you mentioned no brands, but that doesn't really work for me.

P.S. Please don't tell me that "Don sells speakers too" because his array is as much a concept as it is in up and running commecial success.


P.S.S. Your article was fine as an OPINION, but your replies to Krapapple who actually IS a scientist are wanting.

It's ok you don't agree with my article and views as I state at the very beginning:

Before reading this article, I must warn you that it is an opinion piece. You won’t find measurements and analytics from me backing up the statements I make herein. I just felt like venting my views about the abuses of loudspeaker marketing disguised as science and I apologize in advance to any objectivists that will surely take issue with what is written in text below. This article does NOT target any particular manufacturer, but raises issues I have with what I feel to be misconceptions in objective loudspeaker testing and the mindset some manufacturers and consumers tend to have regarding measurements and blind testing.

That being said, I too come from a very strong scientific background. I minored in math in college where I studied calculus based statistics and actually derived the curves and equations that most statisticians use in the field but rarely truly understand. I've worked with actuaries. I worked as a senior member of the Technology group at one of the largest Telecommunications companies studying and analyzing blind data and contributing to peer reviewed standards bodies like T1E1. Most if NOT all of the "DBT Speaker Test" reports I've seen would never hold up to the strictest scientific rigor of a true DBT protocol.

Anyone that blindly accepts a so called "double blind test" procedure or results without questioning the inherent biases of the actual test, cannot really call themselves a scientist. The goal of science is always to disprove itself unlike religion which is to accept on blind faith.

That being said, I have high respect for Don Keele. But like our reviewers (myself included) at some point when he was writing professional reviews, he worked for a magazine. So he is not exempt from the perception of bias because he worked for a "for profit" organization that accepts advertising.

For the record, I don't SELL loudspeakers or A/V products. The Audioholics Estore does and my relationship with them has been made very clear since day one. WE license our name and its a separately run business. I do however sell advertising which I can certainly understand the conflict of interest readers may have but to be fair all review publications sell advertising and we are all in the business of profit else we wouldn't be here.

Let me also point out that the published "double blind tests" are done by companies that SELL speakers. I am not questioning the validity of running a double blind test, but the biases that aren't disclosed or even fully understood in much of the published results.

The abuse of logic some companies use such as "a better speaker cannot be made for more than $1500, and anything higher is paid for cosmetics" is NOT only an insult on ones intelligence, but on the sanctity of those that engineer legitimately higher end products. I find it ironic that I've seen this type of claim made by companies that sell $1500/pair and $15,000/pair speakers. Are we really to assume there are no sonic advantages between their low and high priced speakers if they were to be put them through their DBT procedure? I think any Revel owner would certainly disagree that a pair of Best Buy JBL speakers are "similarly good" and you don't need a DBT to make that conclusion since the sonic differences between the two are quite apparent.

Let me recap some finer points again:

• I am NOT against DBTs or blind tests. We do blind tests. Most people that claim to do DBT's DON'T do DBT's. They at best do SBTs or when using their own staff, simply blind tests.
• Blind tests are NOT infallible nor are they always less biased than sighted tests
• Most loudspeaker measurements are inadequate to determine true product performance and how that performance is perceived by real listeners in a real room

I wonder if I need to restate this at the end of the article as some people in this forum are interpreting my editorial as anti measurements and anti blind tests?

 

[GO-NAD!]

Did AH recommend my speakers?

It would be nice to me.

I like brand name, aesthetics, compliments, nice words, and all that.

I was going to buy the RBH T30, but I just did not like the aesthetics at all.

Actually, you need a pair of 800D's...

 

[AcuDefTechGuy]

Actually, you need a pair of 800D's...

Why.................you.................

Yeah, you and PENG both like to spend my money.

 

[AcuDefTechGuy]

Let me recap some finer points again:

• I am NOT against DBTs or blind tests. We do blind tests. Most people that claim to do DBT's DON'T do DBT's. They at best do SBTs or when using their own staff, simply blind tests.
• Blind tests are NOT infallible nor are they always less biased than sighted tests
• Most loudspeaker measurements are inadequate to determine true product performance and how that performance is perceived by real listeners in a real room

I wonder if I need to restate this at the end of the article as some people in this forum are interpreting my editorial as anti measurements and anti blind tests?

I think so. I have very short term memory.

Back in school, I would raise my hand in class and say, "Sooooooooooooooo, what's the take home message here?"

 

[randyb]

It's ok you don't agree with my article and views as I state at the very beginning:



That being said, I too come from a very strong scientific background. I minored in math in college where I studied calculus based statistics and actually derived the curves and equations that most statisticians use in the field but rarely truly understand. I've worked with actuaries. I worked as a senior member of the Technology group at one of the largest Telecommunications companies studying and analyzing blind data and contributing to peer reviewed standards bodies like T1E1. Most if NOT all of the "DBT Speaker Test" reports I've seen would never hold up to the strictest scientific rigor of a true DBT protocol.

Anyone that blindly accepts a so called "double blind test" procedure or results without questioning the inherent biases of the actual test, cannot really call themselves a scientist. The goal of science is always to disprove itself unlike religion which is to accept on blind faith.

That being said, I have high respect for Don Keele. But like our reviewers (myself included) at some point when he was writing professional reviews, he worked for a magazine. So he is not exempt from the perception of bias because he worked for a "for profit" organization that accepts advertising.

For the record, I don't SELL loudspeakers or A/V products. The Audioholics Estore does and my relationship with them has been made very clear since day one. WE license our name and its a separately run business. I do however sell advertising which I can certainly understand the conflict of interest readers may have but to be fair all review publications sell advertising and we are all in the business of profit else we wouldn't be here.

Let me also point out that the published "double blind tests" are done by companies that SELL speakers. I am not questioning the validity of running a double blind test, but the biases that aren't disclosed or even fully understood in much of the published results.

The abuse of logic some companies use such as "a better speaker cannot be made for more than $1500, and anything higher is paid for cosmetics" is NOT only an insult on ones intelligence, but on the sanctity of those that engineer legitimately higher end products. I find it ironic that I've seen this type of claim made by companies that sell $1500/pair and $15,000/pair speakers. Are we really to assume there are no sonic advantages between their low and high priced speakers if they were to be put them through their DBT procedure? I think any Revel owner would certainly disagree that a pair of Best Buy JBL speakers are "similarly good" and you don't need a DBT to make that conclusion since the sonic differences between the two are quite apparent.

Let me recap some finer points again:

• I am NOT against DBTs or blind tests. We do blind tests. Most people that claim to do DBT's DON'T do DBT's. They at best do SBTs or when using their own staff, simply blind tests.
• Blind tests are NOT infallible nor are they always less biased than sighted tests
• Most loudspeaker measurements are inadequate to determine true product performance and how that performance is perceived by real listeners in a real room

I wonder if I need to restate this at the end of the article as some people in this forum are interpreting my editorial as anti measurements and anti blind tests?

Fair enough. I did try to edit my comments (for whatever reason I could not) as I felt I went a little too far in dismissing your POV. For that I apologize. I do standby the fact that the appearance of a conflict of interest sometimes is just as bad as a true conflict of interest in the minds of the reader. However, I understand the need to sell advertisements. That IS one of the complaints that objectivists have with magazines such Stereophile as you surely know.

I also do not "blindly" accept DBT's however I do accept Toole and Olive's work at the National Canadian Labs and further while I am skeptical of some of the published results of the work at Harmon, I find it interesting and think it is valuable to the end user. For the record, I don't really disagree with anything you said above other than I will continue to seek out information based on blind and double blind tests even if the methods aren't perfect. I also will seek out measurements such as those published by Soundstage done at the Canadian research center. Since it is done by a third party, it counts more with me. I found it interesting that is one of Olive's suggestions to help the industry and that the lab only charges 3-400 to do the test. I wonder why more magazines and mfgs. don't use it. It seems to be a fairly small price to pay for legitimacy.
I did not know your qualifications other than I thought you were an engineer. I am impressed.

 

[kevon27]

Has anyone ever experienced this.. One day your speakers sound awesome (movies and music).. You're surprised how good they some for such a cheap price.. Then the next day (no modification made to anything) they don't sound all that great. And you're asking yourself, are these the same speakers that sounded to good yesterday and now today they ain't all that?

It's weird but it has happened to me. Maybe it's the mood I'm in which causes my perception of sound quality being good or bad to change.

 

[gene]

Fair enough. I did try to edit my comments (for whatever reason I could not) as I felt I went a little too far in dismissing your POV. For that I apologize. I do standby the fact that the appearance of a conflict of interest sometimes is just as bad as a true conflict of interest in the minds of the reader.

I did not know your qualifications other than I thought you were an engineer. I am impressed.

Thanks and quite frankly I've bottled up my opinions from editorials for so long, I felt like I finally wanted to vent my thoughts (correct or not) in an editorial piece and I was expecting a wide reaction like I've seen on this forum.

I try not to stroke my ego by touting my credentials too much. I am always humbled by folks like Dr. Sean Olive & Floyd Toole that dwarf my knowledge and experience in these areas. I enjoy learning from them but I also don't blindly accept published results, even from them, without questions raised in my mind.

 

[Philip Bamberg]

My observation is that many manufacturers don't publish graphical data at all. Or if they do, it is an FR graph which is severely "zoomed out" so as to make the +-dB wiggles appear very small. I mean, the vertical graph might show 80 or 100dB full scale! And this is still absent from brands that are fully capable of developing speakers with complete and valid measurement data and methods.

Anymore, it seems that a specs table is supposed to suffice, listing nominal impedance, bandwidth (FR extremes), and accuracy (+-dB variance from flat).

I do this, too, but then I also include a more comprehensive specs PDF binder which includes graphical data of appropriate resolution. I've even been criticized for the FR to look "rough", that is until the person looks closer at the left-hand scale and realizes the speaker is actually +-2.2dB.

 

[GO-NAD!]

I wonder if I need to restate this at the end of the article as some people in this forum are interpreting my editorial as anti measurements and anti blind tests?

I hope that isn't the impression that I gave you, as that is not what I took from your editorial. It would just be helpful if there was standarized testing of loudspeakers to help narrow down a field of candidates that one could then choose from, to audition at home.

 

[gene]

I hope that isn't the impression that I gave you, as that is not what I took from your editorial. It would just be helpful if there was standarized testing of loudspeakers to help narrow down a field of candidates that one could then choose from, to audition at home.

Believe me I am trying to make this a reality. I lose sleep at night thinking about this. I've spoken to Dr. Floyd Toole and many others in the industry on this very topic. I feel pretty good with how we are reviewing subwoofers and amplifiers/receivers right now and the industry agrees. I want to do the same with loudspeakers but the deeper I dig into this topic, the more complex I realize it is.

Here is my advice to those wanting more clarity on how to interpret data on speakers:

• know how the measurements were made and what the resolution was
• realize that distortion measurements are usually severely lacking in accuracy or in correlation to how the ear perceives distortion. Music is much different than sweep tones.
• recognize if a company claiming "similarly good" or "you can't do better beyond a certain price" that those claims are highly suspicious
• recognize that blind tests can be as biased and flawed as sighted tests
• recognize sighted tests can still be valid if properly controlled
• put more weight on reviewers that take the time to actually measure the products, consult with manufacturers to peer review their results, and give a detailed look inside the box and discuss design theory
• gather user feedback in the forums on the particular product you are interested in assuming they've actually heard the products in question
• trust your instincts, if something seems to good to be true, it probably is
• lastly and most importantly, demo the products for yourself!

I hate to admit this but before I started Audioholics, anytime I was set on buying a new pair of speakers, I would narrow it down to 2-3 models based on in-store demos. I would then purchase all 3 models and bring them home. I would spend weeks comparing them and writing notes. I would also do individual listening tests. I would also bring in my most trusted friends that are audio buffs and collect their opinions. I would then solicit them to help me bring the other two pairs back to the store. By the time I was done with my comparison it was quite clear I chose the right speaker for my needs and I typically lived with them for at least 5-6 years before upgrading again.

 

[wiyosaya]

Has anyone ever experienced this.. One day your speakers sound awesome (movies and music).. You're surprised how good they some for such a cheap price.. Then the next day (no modification made to anything) they don't sound all that great. And you're asking yourself, are these the same speakers that sounded to good yesterday and now today they ain't all that?

It's weird but it has happened to me. Maybe it's the mood I'm in which causes my perception of sound quality being good or bad to change.

Yes. I have experienced this, but perhaps not in the same way.

When I bought my current speakers, in the early 1990's, I auditioned two different pairs from the same manufacturer (Paradigm) at the store. One pair was a "higher end" pair than the other. Both had the same number of drivers, very similar specs, and a very similar response curve.

I brought along several CDs to use as test material. My experience was that some CDs sounded better on the higher end model, and some CDs sounded better on the lower end model. The only difference in the chain was the speakers. Same CD player, same amp - you get the picture.

My choice to purchase was based on this fact, and I chose the lower end model since I did not see the value inherent in the added cost of the higher end model. After all, if either model was just as capable of sounding not as good on different source material, I did not see the point of spending the extra money on the higher end model.

Long story short, source material is as much responsible for the sound coming from those speakers as is speaker design and everything in between.

One other possibility is that maybe if you were sick, had an ear infection, etc., this could also make speakers sound differently.

 

[djreef]

There are a multitude of differnt things that can change, including the thing between your ears that can affect sound and the perception of sound from one day to another. For example, my speakers always sound their best after application of 2 Dewars 12 Year on the rocks.

DJ

 

[lsiberian]

Just in case people mistook my statement I'm not at all opposed to DBT. I've used it as a basis for numerous design decisions and compression decisions. I will say that seeing a speaker does help me in evaluating it because I can tell a lot by looking at a speaker, but good sound is no excuse for ugly IMO. My wife may differ on that statement though.

I don't think a loudspeaker testing standard is impossible Gene. I think it's very expensive to pull off. I'd advise 3 placements for each speaker(on wall, 3ft, 6ft) in a typical living room settting, home theater setting and bedroom setting.

I wish you best of luck in this endeavor. It's not easy.

 

[Josuah]

Thanks for posting this editorial. I've been slightly turned off to the Audioholics community as a whole because of the whole "measurements" and "DBT" and "amps are all the same" vocal members.

The existing subset of published measurements are not sufficient to completely characterize the audio quality of a product. Large scale statistical testing may matter for a population at large, but not necessarily for an individual.

And thank you also for explicitly mentioning detailed can be the same as distortion and smooth the same as veiled. You have no idea how many people I run into thinking that something is wrong when this "stuff" is missing from a high-end setup, especially when very expensive gear also exhibits these characteristics.

 

[highfigh]

Tread carefully Gene. You start talking sense and the crazies will come out of their crazy rooms.

Looks matter. Good finishes inspire confidence and improve the listening experience. Why shouldn't we allow sight to influence our perception? Are we supposed to be robots?

"What kind of acoustic treatments do I need in my crazy room?"

Signed,
Confused

Cosmetics make us like the way our speakers look, but that is all. If some company can make speakers that are ugly (several do) but they sound better than the kind that are all sparkly and sound good/not great, why buy the pretty ones? Speakers are for listening to, after all. If not for WAF or guys who actually care about cosmetics, they could all be painted a neutral color and it would be OK. The cost of exotic woods and other materials doesn't make them sound better, so why use them?

Sex sells.