Speaker Reviews, a Critique

[Tomorrow]

Since joining this forum, I have learned much about the a/v world and I am much appreciative of all of you contributors...and even the lurkers who keep this place going and help pay the bills, so to speak. There is a nice mix of attitudes, beliefs, approaches, hard data, opinions, bs, deep knowledge, and lots of fun and funny people. The few bad seeds usually weed themselves out, but many thanks to our moderators for helping them along. So....my congrats to you all for putting up an interesting and all too distracting (according to the missus, anyway ) place to visit. Happy 2007 to you all.

Now that I have you all sweet-talked, lol, it's time to stir the pot again, as seems to be my way. I'd like your opinions on the following matter....SPEAKER REVIEWS in general.

Since joining here a couple of years ago and attempting to build my knowledge base, I've probably read hundreds of speaker reviews by both professional (industry involved) and hobbyists alike, here and in other publications. To my best recollection, none of them, with the one exception of Craigsub's shootout last year, involved direct A/B testing, or better yet, full-blown DBT evaluation. We seem to demand this kind of data in all our audio products, except speakers. We tend to banty about descriptive phrases about types of speakers, but using only subjective imaginings or memories to call upon.

As I audition, and now own a variety of speakers, I've found it most interesting that I'll react one way to a speaker's presentation....and entirely differently when I evaluate it in a side-by-side with another brand/model. (Many thanks to AverageJoe for providing assistance and some of the gear that gets used in these speaker evaluations...not the least of which is his contribution of labor and beautiful, custom built speaker cables.) It has become abundantly clear, to me anyway, that speakers simply cannot be evaluated or properly reviewed without comparing their presentation with another (or several) competitors...in an A/B or DBT fashion. Gene did a thorough bookshelf speaker comparison review, and perhaps even a couple of others. But, with Craigsub's comparo aside, I don't ever recall a DBT or even A/B switching to aid with the development the adjectives we all like to provide when describing the sound of a speaker. While I'm sure this has occurred within the industry or on other boards, it just is not the common practice that it seems it should be.

When you go out to a store to audition speakers, even though it's not in your own listening environment, the store will generally do fast speaker switching for you, allowing you the kind of listening that I think makes the most sense when selecting something to buy. When this approach is taken, one can give relative meaning to terms like 'warm', or 'detailed', or 'airy', etc. I know that many of the 'pros' have distinguished careers in field and can pick out speaker details from experience and evaluate the items with much greater accuracy than I. But still, it would mean much more to me, the reader, if I knew the review writer was DBT'ing the units with other, known speaker elements and was attempting to find some standardization of the language and comparability of speaker sonance.

What do you all think?

 

[mike c]

you mean the statements:

"this speaker is the best in its class between 399-400USD"
"this speaker can beat even those that cost 5 times more"
"this speaker is the greatest ever, but still cant beat my 'Exorbitant' speakers"
"this speaker is ultra smooth like chocolate syrup ... but my 'Exorbitant' speakers are like maple syrup"

aren't clear enough?

i've pretty much stopped reading subjective speaker reviews.
as much as I agree with what you want to happen ... it might not be possible due to the highly subjective tastes of people.
I'm considering buying another pair of bookshelves so I can A/B them with my Polks myself. (choosing between the Yamaha NS-333 and Monitor Audio BR1)

 

[Audioholics]

Personally, I'd love to do it if nothing else was a factor. What we run into are issues of time, setup, and the fact that almost no manufacturers want their products reviewed in this manner - and for good reason. These dbt's and multi-speaker reviews are almost never done "correctly" - even craigsub will probably tell you his setup isn't ideal and that his conclusions are highly subjective at best.

For a true dbt to be accomplished you need:

• a fixed room with decent (measured) acoustics
• a way to easily and quickly locate all speakers to the same exact location (setting them up side by side is just asking for trouble in terms of comparing imaging, soundstage and low-end response) - i.e. a Harmon-style lazy susan
• a method of running the ABX test accurately (procedure as well as hardware)
• a panel of trained or untrained listeners who can assemble together and agree upon terminology and a set of listening critieria as well as results quantification. (i.e. if one of them doesn't understand "soundstage" it's hard to rate on a scale of 1-10.)

So you can see the issues - and why most manufacturers specifically want to know if this is going to be done before submitting their equipment for review...

Bottom line - it's a pain in the butt... When we do subjective reviews, we have certain "control" speakers that we use as a baseline reference. That guides our opinions on anything else and keeps us "grounded" in what the room is doing to the sound. My reference pair, for example, are the RBH Sound 61-SE/Rs. You can find better and worse speakers than those for certain, but I know how they are supposed to sound and I've heard them in a total of 4 different rooms for reference.

Each reviewer is also different and most people seem to learn the "tastes" and preferences of each. A good example is the local restaurant critic in Orlando - he's awesome and I know that any restaurant he likes is going to be good. The idiots in Myrtle Beach, on the other hand, thought Olive Garden was the epitome of Italian retaurants.

 

[Joe Schmoe]

While A/B comparison is ideal, it is also frequently impossible. If I think of even two (much less three or more) similar-priced speakers that I would like to compare, it is virtually certain that they are in separate stores (if they are available locally at all.) Someone rich enough could buy both to compare at home and return the "loser", I guess, but most people don't have that kind of spare cash laying around.
I usually start with the manufacturer's reputation, the design, the technologies used, user reviews, etc. to narrow the list. After that I buy the ones within my budget that impress me the most at the time I am auditioning them.

 

[Seth=L]

I find it strange that almost all the reviews I read about speakers I have, they all sound almost the same. The speaker is always the best the reviewer has heard in that price range. That is just grade A bologna.

 

[no. 5]

I find it strange that almost all the reviews I read about speakers I have, they all sound almost the same. The speaker is always the best the reviewer has heard in that price range. That is just grade A bologna.

"we all need a well-equipped Baloney Detection Kit."
-Carl Sagan

 

[Joe Schmoe]

I find it strange that almost all the reviews I read about speakers I have, they all sound almost the same. The speaker is always the best the reviewer has heard in that price range. That is just grade A bologna.

To say that a speaker is merely the best in its price range is almost an insult, considering how many are "better than speakers costing three times as much".

 

[gmichael]

So I shouldn't believe the following?

1) These $500 speakers are better than any of the $1000 speakers on the market.
2) These $1000 speakers are better than any of the $5000 speakers on the market.
3) These $5000 speakers are better than any of the $10,000 speakers on the market.
4) These $10,000 speakers are better than any of the $25,000 speakers on the market.
etc..........

N!) So, the $500 speakers in number 1 are better than any of the 1/4 of a million dollar speakers available anywhere? They sound nice and airy, clear, natural and image better than anything ever made?

Darn! And I thought I could get something for nothing.

Sorry, just havin' some fun here.

 

[edonsmith]

I too would love to see this kind of testing/comparisons.

I would also like to see blind testing where the listener doesnt know what he is listening to

Don

 

[STRONGBADF1]

Personally, I'd love to do it if nothing else was a factor. What we run into are issues of time, setup, and the fact that almost no manufacturers want their products reviewed in this manner - and for good reason. These dbt's and multi-speaker reviews are almost never done "correctly" - even craigsub will probably tell you his setup isn't ideal and that his conclusions are highly subjective at best.

For a true dbt to be accomplished you need:

• a fixed room with decent (measured) acoustics
• a way to easily and quickly locate all speakers to the same exact location (setting them up side by side is just asking for trouble in terms of comparing imaging, soundstage and low-end response) - i.e. a Harmon-style lazy susan
• a method of running the ABX test accurately (procedure as well as hardware)
• a panel of trained or untrained listeners who can assemble together and agree upon terminology and a set of listening critieria as well as results quantification. (i.e. if one of them doesn't understand "soundstage" it's hard to rate on a scale of 1-10.)

So you can see the issues - and why most manufacturers specifically want to know if this is going to be done before submitting their equipment for review...

Bottom line - it's a pain in the butt... When we do subjective reviews, we have certain "control" speakers that we use as a baseline reference. That guides our opinions on anything else and keeps us "grounded" in what the room is doing to the sound. My reference pair, for example, are the RBH Sound 61-SE/Rs. You can find better and worse speakers than those for certain, but I know how they are supposed to sound and I've heard them in a total of 4 different rooms for reference.

Each reviewer is also different and most people seem to learn the "tastes" and preferences of each. A good example is the local restaurant critic in Orlando - he's awesome and I know that any restaurant he likes is going to be good. The idiots in Myrtle Beach, on the other hand, thought Olive Garden was the epitome of Italian retaurants.

Hi Clint,

This could be done at a once a year get together where members would bring their own speakers. (all set-up and approved by AH of course.)

SBF1

p.s. we could also do a restuarant review while we're at it.

 

[cyfman]

i agree with what everyone has said,each person has his own views on things.i do more lurking than posting because all our tastes and ears are different.i've done it myself ask what someone would choose between speaker A or B.so unless a lot of people could get together and bring their own speakers as another poster said,it's almost impossible imo to say get this or that brand of speakers and this speaker company is junk. but that is what makes this "hobby" fun and enjoyable. so everyone keep up the good work and have a safe and happy new year.

 

[Seth=L]

I have never seen a review that said a lower priced speaker sounded better than all speakers that cost twice the amount.

Instead you see phrases like this.

"Better than other speakers I have listened to that cost twice as much".

They just make it sound like they are the best, but they are being vague and non-obsolute. Of course no one can say what will sound better from person to person.

 

[Buckle-meister]

Since joining this forum, I have learned much about the a/v world and I am much appreciative of all of you contributors...and even the lurkers who keep this place going and help pay the bills, so to speak. There is a nice mix of attitudes, beliefs, approaches, hard data, opinions, bs, deep knowledge, and lots of fun and funny people. The few bad seeds usually weed themselves out, but many thanks to our moderators for helping them along. So....my congrats to you all for putting up an interesting and all too distracting (according to the missus, anyway ) place to visit. Happy 2007 to you all.

Now that I have you all sweet-talked, lol, it's time to stir the pot again, as seems to be my way. I'd like your opinions on the following matter....SPEAKER REVIEWS in general.

Since joining here a couple of years ago and attempting to build my knowledge base, I've probably read hundreds of speaker reviews by both professional (industry involved) and hobbyists alike, here and in other publications. To my best recollection, none of them, with the one exception of Craigsub's shootout last year, involved direct A/B testing, or better yet, full-blown DBT evaluation. We seem to demand this kind of data in all our audio products, except speakers. We tend to banty about descriptive phrases about types of speakers, but using only subjective imaginings or memories to call upon.

As I audition, and now own a variety of speakers, I've found it most interesting that I'll react one way to a speaker's presentation....and entirely differently when I evaluate it in a side-by-side with another brand/model. (Many thanks to AverageJoe for providing assistance and some of the gear that gets used in these speaker evaluations...not the least of which is his contribution of labor and beautiful, custom built speaker cables.) It has become abundantly clear, to me anyway, that speakers simply cannot be evaluated or properly reviewed without comparing their presentation with another (or several) competitors...in an A/B or DBT fashion. Gene did a thorough bookshelf speaker comparison review, and perhaps even a couple of others. But, with Craigsub's comparo aside, I don't ever recall a DBT or even A/B switching to aid with the development the adjectives we all like to provide when describing the sound of a speaker. While I'm sure this has occurred within the industry or on other boards, it just is not the common practice that it seems it should be.

When you go out to a store to audition speakers, even though it's not in your own listening environment, the store will generally do fast speaker switching for you, allowing you the kind of listening that I think makes the most sense when selecting something to buy. When this approach is taken, one can give relative meaning to terms like 'warm', or 'detailed', or 'airy', etc. I know that many of the 'pros' have distinguished careers in field and can pick out speaker details from experience and evaluate the items with much greater accuracy than I. But still, it would mean much more to me, the reader, if I knew the review writer was DBT'ing the units with other, known speaker elements and was attempting to find some standardization of the language and comparability of speaker sonance.

What do you all think?

I agree!

 

[mtrycrafts]

"we all need a well-equipped Baloney Detection Kit."
-Carl Sagan

Oh, so you read him too

He was good at that kit and not being left at the office all the time

He also commented about 'open minds' in his speech in Pasadena many years back.

 

[mtrycrafts]

...i do more lurking than posting because all our tastes and ears are different..

Actually, while we may like different music, etc, we do prefer a very much similar sound from speakers:

http://miragespeakers.com/nrc_story.shtml

Based on the results of the NRC’s research, Mirage engineers determined that there were three characteristics common to the speakers that consistently achieved high scores in the listening trials. It was determined that listeners preferred speakers that could produce the whole range of frequencies (Wide Bandwidth), equally efficiently (Flat Response), both on and off axis (Wide Dispersion), clearly (No Distortion).

• Ian Paisley and Andrew Welker are the designers of Mirage speakers.
• Ian was involved in the famous National Research Council speaker tests.
• These tests determined that the characteristics that people preferred in speakers were:
- Low Distortion
- Flat Response and wide bandwidth
- Wide Dispersion

 

[Tomorrow]

Based on the results of the NRC’s research, Mirage engineers determined that there were three characteristics common to the speakers that consistently achieved high scores in the listening trials. It was determined that listeners preferred speakers that could produce the whole range of frequencies (Wide Bandwidth), equally efficiently (Flat Response), both on and off axis (Wide Dispersion), clearly (No Distortion).

• Ian Paisley and Andrew Welker are the designers of Mirage speakers.
• Ian was involved in the famous National Research Council speaker tests.

And this is why Mirage Speakers are so wildly popular today. LOL.

 

[no. 5]

Oh, so you read him too

well, I like quotes.

 

[mtrycrafts]

And this is why Mirage Speakers are so wildly popular today. LOL.

Well, you know about audiophiles, name matters as does looks. Heaven forbid one would select by DBT listening only. Half the audio makers would be history, or more.

 

[Tomorrow]

Well, you know about audiophiles, name matters as does looks. Heaven forbid one would select by DBT listening only. Half the audio makers would be history, or more.

It also doesn't speak well for the transition of the NRC studies to manufacturer implementation. And there is a possibility that the NRC studies are flawed. I haven't read the methodology used, but it was done in Canada, after all.

 

[mtrycrafts]

And there is a possibility that the NRC studies are flawed. I haven't read the methodology used, but it was done in Canada, after all.

Yes, they could be flawed, but then you'd have the burden to show this to be the case.
Unlike speaker companies that don't use similar protocols of the NRC, I'll take the companies that do any day of the week. Including Sundays.