The Insanity of Marketing Disguised as Science in Loudspeakers

highfigh

highfigh

Seriously, I have no life.
Believe me I am trying to make this a reality. I lose sleep at night thinking about this. I've spoken to Dr. Floyd Toole and many others in the industry on this very topic. I feel pretty good with how we are reviewing subwoofers and amplifiers/receivers right now and the industry agrees. I want to do the same with loudspeakers but the deeper I dig into this topic, the more complex I realize it is.

Here is my advice to those wanting more clarity on how to interpret data on speakers:
  • know how the measurements were made and what the resolution was
  • realize that distortion measurements are usually severely lacking in accuracy or in correlation to how the ear perceives distortion. Music is much different than sweep tones.
  • recognize if a company claiming "similarly good" or "you can't do better beyond a certain price" that those claims are highly suspicious
  • recognize that blind tests can be as biased and flawed as sighted tests
  • recognize sighted tests can still be valid if properly controlled
  • put more weight on reviewers that take the time to actually measure the products, consult with manufacturers to peer review their results, and give a detailed look inside the box and discuss design theory
  • gather user feedback in the forums on the particular product you are interested in assuming they've actually heard the products in question
  • trust your instincts, if something seems to good to be true, it probably is
  • lastly and most importantly, demo the products for yourself!

I hate to admit this but before I started Audioholics, anytime I was set on buying a new pair of speakers, I would narrow it down to 2-3 models based on in-store demos. I would then purchase all 3 models and bring them home. I would spend weeks comparing them and writing notes. I would also do individual listening tests. I would also bring in my most trusted friends that are audio buffs and collect their opinions. I would then solicit them to help me bring the other two pairs back to the store. By the time I was done with my comparison it was quite clear I chose the right speaker for my needs and I typically lived with them for at least 5-6 years before upgrading again.
In a store, or even at home, do you tend to close your eyes or turn the lights off when you begin listening to a new speaker?
 
gene

gene

Audioholics Master Chief
Administrator
In a store, or even at home, do you tend to close your eyes or turn the lights off when you begin listening to a new speaker?
In my theater room, I usually dim the lights. When I do critical evaluations, I usually close my eyes until I decide to write something on paper. I find I can concentrate better with low lighting and my eyes closed.
 
djreef

djreef

Audioholic Chief
In my theater room, I usually dim the lights. When I do critical evaluations, I usually close my eyes until I decide to write something on paper. I find I can concentrate better with low lighting and my eyes closed.
Ah, that dreaded X factor for which no measurement can detect. Individual Perception.

DJ
 
gene

gene

Audioholics Master Chief
Administrator
Article Addendum

Folks;

I just added this to the end of the article. I hope it clears up some misunderstandings people may have had by reading my article.

Article Addendum

added: 2/10/12

I thought it might help to recap the main points in this article for the forum folks that may have misunderstood my intentions of this opinion piece.

  • We are NOT anti-measurement or anti-DBT / anti-blind testing.
  • We engage in our own blind test protocols frequently when doing product comparisons.
  • Recognize that most companies that claim to use DBT protocol don't. At best they are doing a SBT or if using their own staff, a highly biased blind test.
  • Recognize if a company claiming "similarly good" or "you can't do better beyond a certain price" that those claims are highly suspicious.
  • Recognize that blind tests can be as biased and flawed as sighted tests.
  • Recognize sighted tests can still be valid if properly controlled.
  • Put more weight on reviews and reviewers that take the time to actually measure the products, consult with manufacturers to peer review their results, and give a detailed look inside the box and discuss design theory.
  • Gather user feedback in the forums on the particular product you are interested in assuming said users actually heard the products in question.
  • Recognize that all measurement techniques and reports aren't created equal.
  • Not everything we can measure relates to audibility and not everything that is audible can be directly measured in loudspeakers.
  • Realize that distortion measurements are usually severely lacking in accuracy or in correlation to how the ear perceives distortion. Music is much different than sweep tones.
  • Trust your instincts, if something seems to good to be true, it probably is regardless of how much "science" is wrapped around it.
  • Lastly and most importantly, demo the products for yourself, preferably with familiar listening material and in your own listening environment.
 
W

wiyosaya

Audioholic
You have no idea how many people I run into thinking that something is wrong when this "stuff" is missing from a high-end setup, especially when very expensive gear also exhibits these characteristics.
Because of societal conditioning, I think many people equate "performance" and "price" and expect that a high price must mean high performance. IMHO, high-performance does not necessarily mean a high price, nor does a high price mean high performance.

Personally, I think anyone focusing on having gotten the most expensive piece of equipment from any manufacturer is likely missing the point and more interested in the "trophy factor" than in truly good equipment. As well, I think that many items that command top dollar rely on an undeserved mythical image their products are unequaled for their sales.

Therefore, I take people who flaunt what they paid for something with a large grain of salt, and I usually end up ignoring them. What I think is most important is to audition the equipment in an as controlled as possible way and make a decision to buy based on that audition. As this article indicates, specs and even design philosophies do not tell the whole story.
 
F

fredk

Audioholic General
Gene. I think that is a worth while addendum. I'm afraid I too came away with the impression you really didn't like DBTs.

I had the pleasure of participating in a properly set up DBT between a modestly priced and very expensive speaker. I found it quite difficult to express my views on the two speakers as they really were "similarly good", though not the same. It took a long time to recognize where a difference was coming from, in part because program material is so varied and that affects how/when things stand out.

I doubt I would be able to set up such a test in home. It would take proper equipment and a lot of effort.

As far as measurements and understanding what they represent, not many people are going to invest the time and effort to fully understand what they are looking at. Heck, I've spent a lot of time and still don't fully understand how you combine different measurements to look for things (that may or may not affect my impression of a speaker).

There is one thing I don't get. If you are OK with Bose using marketing fluff to sell, why are you not OK with other companies using marketing fluff (see the pretty flat line?) to sell. Representing your product in 'the best light' is no different whether you use Bose speak or smoothed on axis sweeps, at least not to me.
 
Irvrobinson

Irvrobinson

Audioholic Spartan
Has anyone ever experienced this.. One day your speakers sound awesome (movies and music).. You're surprised how good they some for such a cheap price.. Then the next day (no modification made to anything) they don't sound all that great. And you're asking yourself, are these the same speakers that sounded to good yesterday and now today they ain't all that?

It's weird but it has happened to me. Maybe it's the mood I'm in which causes my perception of sound quality being good or bad to change.:confused:
The same thing happens to me quite a bit. Sometimes it's the background noise (like a windy day), sometimes my allergies get my ears a little stuffed (which I've noticed dulls my hearing in the 100-200Hz range), or my mood is wrong. Whatever. The system is the same, the problem is that humans are fragile wetware.
 
Irvrobinson

Irvrobinson

Audioholic Spartan
My Salon2 sound freaking awesome, though. How about yours?:D
No guessing about that. Mine too.

I like what Gene has written. Good stuff. But I still think audio DBTs are pretty much useless with great equipment.
 
gene

gene

Audioholics Master Chief
Administrator
Gene. I think that is a worth while addendum. I'm afraid I too came away with the impression you really didn't like DBTs.

I had the pleasure of participating in a properly set up DBT between a modestly priced and very expensive speaker. I found it quite difficult to express my views on the two speakers as they really were "similarly good", though not the same. It took a long time to recognize where a difference was coming from, in part because program material is so varied and that affects how/when things stand out.

I doubt I would be able to set up such a test in home. It would take proper equipment and a lot of effort.

As far as measurements and understanding what they represent, not many people are going to invest the time and effort to fully understand what they are looking at. Heck, I've spent a lot of time and still don't fully understand how you combine different measurements to look for things (that may or may not affect my impression of a speaker).

There is one thing I don't get. If you are OK with Bose using marketing fluff to sell, why are you not OK with other companies using marketing fluff (see the pretty flat line?) to sell. Representing your product in 'the best light' is no different whether you use Bose speak or smoothed on axis sweeps, at least not to me.
No I more or less don't like the abuse of the term DBT. I went in and tweaked my language a bit for clarity. I encourage all that took issue with my article to please pain thru re-reading it and tell me if it reads better now.

I never said I was OK or content with Bose. I respect/admire their marketing and business model however. They are brilliant.

Here is what Bose doesn't do that some high end loudspeaker companies do:
  • they DON'T publish measurements
  • they DON'T publish specs
  • they DON'T publish papers pretending to be DBT's
  • they don't declare their speakers are "similarly good" to the very best and elaborate designs
  • they DON'T target audiophiles

Bose mostly compares their product to what appears to be BIG white Van type rack systems. In most cases, their stuff will sound better than the alternatives they show in their ads and their stuff is a lot more compact and easier to use.
 
jliedeka

jliedeka

Audioholic General
Audio perception is a funny thing. My L/R speakers are Jamo C605s that I've had for a few years. They were a huge step up from the low end Klipsch monitors I had before, even with a subwoofer.

About a year ago, I put in a CD. I wish I could remember which one, might have been Shaman's Harvest. Suddenly I realized the midrange didn't sound right. I toed the speakers in 10 or 15 degrees which solved the problem. I was living happily with speakers that blew away my old ones but it took a certain piece of music to draw my attention to a problem.

I'm not saying there isn't any value to critical listening because my example shows there is. It can be hit or miss, however. Before noticing the problem, I had listened to many well recorded discs of classical and jazz music. It was a rock CD that tipped me off.

Jim
 
TLS Guy

TLS Guy

Seriously, I have no life.
No I more or less don't like the abuse of the term DBT. I went in and tweaked my language a bit for clarity. I encourage all that took issue with my article to please pain thru re-reading it and tell me if it reads better now.

I never said I was OK or content with Bose. I respect/admire their marketing and business model however. They are brilliant.

Here is what Bose doesn't do that some high end loudspeaker companies do:
  • they DON'T publish measurements
  • they DON'T publish specs
  • they DON'T publish papers pretending to be DBT's
  • they don't declare their speakers are "similarly good" to the very best and elaborate designs
  • they DON'T target audiophiles

Bose mostly compares their product to what appears to be BIG white Van type rack systems. In most cases, their stuff will sound better than the alternatives they show in their ads and their stuff is a lot more compact and easier to use.
This is a good thread Gene, and I agree with your sentiments. It has taken a while to get through this thread.

I have a number of observations about speaker evaluation, and some are understandably controversial.

First DBT of speakers is nothing like DBT in medicine.

In the years of rapid pharmaceutical progress powering a study was easy. For instance it did not require many subjects to get a statistically significant result for treating pneumococcal pneumonia.

However as effective treatments for many problems progressed, it became important to find benefits of new treatments over old. In order to find small benefits of A over B enormous numbers of patients have to be often involving different centers world wide.

Now to make the data meaningful you have to minimize the variables among the enrolled patients. Otherwise you have to many confounding variables.

Now the end points are definite, such as death, strokes, bleeding episodes, days in ICU, all kinds of definite endpoints.

However, so many studies run on the rocks because of necessary inclusion and exclusion criteria. So endless arguments rage of selection bias and also about drop outs. There are frequent arguments over statistical methods. Don't forget we are dealing with definite identifiable end points as a rule, and not subjective issues like the sound of a speaker.

Now with speakers there are usually large easily identifiable differences between speakers, unlike other parts of the audio chain. So this helps.

However, I think testing speakers with other than highly trained listeners with well developed acoustic memories means nothing at all: - zero.

In my opinion only natural instruments in a known acoustic space can be used for loudspeaker evaluation.

In other words, the panel need to have heard musicians in a space in which they were recorded with the same music in a proximate time period. I firmly believe anything else is useless.

Pop and electronic instruments are hopeless for speaker evaluation.

That is why I use recordings made in a venue I'm familiar with for critical evaluation. If there is a CD available were I have heard a concert I buy it.

When I was at Mahtomedi, MN for the concert celebrating 30 years of Pipe Dream broadcasts I bought a well recorded CD on the Pro Organo label.

The main reason that I made so many recordings over the years, was to evaluate speakers. To go from auditorium to the control room and hear it over my monitor speakers, really developed in me an enormously heightened awareness for the evils of speakers, and a steady improvement in results.

Measurements get you some of the way, but all the good designers I have known voice to a very large extent by ear. The good ones make changes to the design over long intervals after first getting the speaker reasonably balanced.

You are right that power handling is a huge confounding variable not addressed in almost all measurement programs, but it is among the most important factors in the design of a speaker, especially for classical music.

Recently I took measurements while listening to a very impressive BD release of the Mahler symphony No 5. There were prolonged quiet passages were the SPl was below the low range of 50 db. There was a lot between 65 and 85 db, with loud fortissimo peaks going to 102 db at least at the listening position. Now that presents a huge problem to the speaker, and drivers need to be underslung with good heat transfer from the VC among other attributes.

Now we get to an area others have touched on. That is the pretty much impossible task, in the US in particular, of being able to audition a wide variety of speakers. This is especially true at the upper end of the market.

This leads me to believe that the surest way to good sound in the home is to learn to design and build your own speakers. I'm unusual, in that I have never owned a set of commercial speakers. I have always listened to speakers I designed and built. So is there a selection for a particular
sound field? Yes, there must be. Friends and acquaintances have said my speakers have a definite stamp to them, which is usually described as full bodied with a very tight bass. Musicians in particular note the correct space around instruments. I think they mean by that, that the sound does not seem to emanate form the speakers, and the cues from the original acoustic space are preserved. They seem to like them. In Grand Forks we had a listening group and we all had very decent speakers. We would rotate venues, but overtime it seemed to drift more and more often to my place. I think that it all comes down to voicing to the space among other things. I believe speakers like organs have to be voiced to their space.

When I was recording, by monitors had switches on the back to subtley voice them to my most frequent monitoring venues, such as the Green Room at the Chester Fritz auditorium Grand Forks.

The above is yet another inducement to build ones own speakers. The down side is years of progress, but it is a fascinating journey, to Peter Walkers "Closest Approach to the Original Sound." That really has to be the goal of acoustic engineering.
 
Last edited:
P

PENG

Audioholic Slumlord
In my opinion only natural instruments in a known acoustic space can be used for loudspeaker evaluation.

In other words, the panel need to have heard musicians in a space in which they were recorded with the same music in a proximate time period. I firmly believe anything else is useless.

Pop and electronic instruments are hopeless for speaker evaluation.
You said 'opinion', but to me, close enough to being:D 'facts'.

That is why I use recordings made in a venue I'm familiar with for critical evaluation. If there is a CD available were I have heard a concert I buy it.
Great idea, but it must be hard to do and I would consider myself lucky if I can own even just a couple of such CD. It would be a little easier for people who frequent concerts.
 
B

bogrod

Junior Audioholic
[*] Recognize that most companies that claim to use DBT protocol don't. At best they are doing a SBT or if using their own staff, a highly biased blind test.

[*] Recognize if a company claiming "similarly good" or "you can't do better beyond a certain price" that those claims are highly suspicious.
Smiled when I read these two points, because both in your original article - and addendum - I kept on thinking of Axiom. :)
 
Rickster71

Rickster71

Audioholic Spartan
We explore many of the misconceptions consumers often fall victim to when viewing loudspeaker measurements or falling too heavily into the DBT mantra.
Great article Gene. I just had to quote my favorite line.

I see thread after thread of people touting speaker measurements. Never once do they consider getting their own hearing measured.
All of us vary so much in our physical make-up, abilities and senses. Everything from blood type, shoe size, visual acuity....and yes, our hearing.
It's out of the realm of possibility, that us flawed humans, can all hear or appreciate a speaker that measures perfectly.
 
djreef

djreef

Audioholic Chief
So, why doesn't somebody (for the sake of argument) record several live musicians one at a time in a room using a digital recorder (hard drive) in front of the test subject, then immediately play that same recording back through the speaker to be tested? With current technology this should be more than possible. If nothing else you'd be able to test for realism and accuracy of timbre, and to a degree you'd be able to account for whatever time delay issues. I dunno, just trying to think outside the box. I think all of this is kind of pointless stuff anyway with so many subjective intangibles working against you.


DJ
 
H

Hocky

Full Audioholic
So, why doesn't somebody (for the sake of argument) record several live musicians one at a time in a room using a digital recorder (hard drive) in front of the test subject, then immediately play that same recording back through the speaker to be tested? With current technology this should be more than possible. If nothing else you'd be able to test for realism and accuracy of timbre, and to a degree you'd be able to account for whatever time delay issues. I dunno, just trying to think outside the box. I think all of this is kind of pointless stuff anyway with so many subjective intangibles working against you.


DJ
This kind of thing happens occassionally at audio shows and then only thing that you'll realize is that recorded music sounds absolutely nothing like live music, even in the same room, minutes later, from a six figure system.
 
Irvrobinson

Irvrobinson

Audioholic Spartan
So, why doesn't somebody (for the sake of argument) record several live musicians one at a time in a room using a digital recorder (hard drive) in front of the test subject, then immediately play that same recording back through the speaker to be tested? With current technology this should be more than possible. If nothing else you'd be able to test for realism and accuracy of timbre, and to a degree you'd be able to account for whatever time delay issues. I dunno, just trying to think outside the box. I think all of this is kind of pointless stuff anyway with so many subjective intangibles working against you.


DJ
One test I did, when I was using Legacy Audio Focus speakers, was that I played back a studio recording of my step daughter playing solo flute while she played along with herself, standing between the speakers. Of course, my family room wasn't the same venue as the recording was made in, and a flute isn't exactly a difficult instrument for expensive loudspeakers to reproduce, but it was still interesting.

Just for starters, live instruments played at a close distance in a home-sized venue are louder than you might think. To make the recorded flute have the same volume as the live flute had me turning up the volume control more than I normally did. I measured peaks at my listening seat in the 95+db range, c-weighted. Once levels were matched it was still easy to tell the difference between the live playing and the recording, because the Legacys were good enough to let me discern the difference in sound between the venues. But the interesting thing was the recording sounded almost as "live", just different.

Even my latest system can't come close to reproducing my wife's drum kit, even the small jazz kit. Even at 10 feet from the drums she's hitting peaks over 110db when she gets enthusiastic.
 
Irvrobinson

Irvrobinson

Audioholic Spartan
This kind of thing happens occassionally at audio shows and then only thing that you'll realize is that recorded music sounds absolutely nothing like live music, even in the same room, minutes later, from a six figure system.
"Absolutely nothing like live music..."? Been there, done that, you're exaggerating. Not quite like, is more like it.
 
djreef

djreef

Audioholic Chief
"Absolutely nothing like live music..."? Been there, done that, you're exaggerating. Not quite like, is more like it.
I could imagine the difficulty with capturing transients and dynamic attack accurately. Hell, these are paradigms that have been baffling engineers since the beginning of recorded history, but with 24/192 I'm thinking we should at least be close. I mean TLS has argued that 16 bit in the right hands should be close enough to fool most folks from the point of reproduction of dynamic range, but that doesn't bear out. I'm thinking most people could tell the difference quite easily between a live solo instrument, or voice and it's reproduction, even through a mega-rig.

So, the argument is from the hardware side, then. The equipment hasn't progressed enough to realistically reproduce with enough accuracy to fool someone into actually thinking they're listening to a live musician through reproduced media at sound pressure levels that approximate a live event.

DJ
 
newsletter

  • RBHsound.com
  • BlueJeansCable.com
  • SVS Sound Subwoofers
  • Experience the Martin Logan Montis
Top