Why are Speaker Review so bad?

[Hobbit]

It seems the majority of speaker reviews I've read are close to pointless.

I believe reviewing speakers should take the form similar to that of how car mags review cars. For instance, a shootout with 4 speakers that are similar in price and target audience. First, make measurements to see if they meet their published specs. Second, take 4 reviewers to subjectively review them by running them through their paces with various music types and even movies. Finally, compare the specs and contrast their strengths and weaknesses. Including the subjective ones. With car review you'll often read how one driver didn't find a seat comfortable.

I'll bet most $1000 (or $5000) dollar speakers will end up performing more similar than different. If one is that bad, call them on it. Conversely, if one is that much better, point it out as the best in class.

This brings me to one last point. Why is it virtually every review I've read on ~$1000 speaker the reviewer insists on comparing it to a speaker costing 5-6 times more? Usually with a comment along the lines of you wouldn't expect that performance at that level. Is this a badge of honor? Bragging rite? I don't get it. I only want to read a comment like this if the $1000 speaker is performing like a $2000 speaker. Beyond that it's a pointless comment. Moreover, these types of comments and comparisons are very rare in car reviews. Do they really need to say the Miata is slower than a 911 Turbo that costs 5-6x more?

The reviews won't be perfect, but at least they'll be much better than they are now.

Can you tell I've been getting the itch to upgrade;-)

 

[Seth=L]

Unlike cars speakers are purely subjective. Sure, you'll still get the die hards that argue which muscle car is better even after being shown factual specifications and statistical performance and there are plenty of factors to consider but speakers are almost entirely subjective. What sound I like out of a speaker you might find quite distasteful and likewise I might like the sound you like. Expecting to find out how much you will like a speaker is from a review is like expecting to know how much you'll like a music album based on a review, especially if you've never heard the performer.

 

[Sheep]

Unlike cars speakers are purely subjective. Sure, you'll still get the die hards that argue which muscle car is better even after being shown factual specifications and statistical performance and there are plenty of factors to consider but speakers are almost entirely subjective. What sound I like out of a speaker you might find quite distasteful and likewise I might like the sound you like. Expecting to find out how much you will like a speaker is from a review is like expecting to know how much you'll like a music album based on a review, especially if you've never heard the performer.

True, but honest measurements will tell you what it sounds like. Once you establish your preference, a reliable set of measurements gets you a good understanding of how the speaker performs.

SheepStar

 

[Hobbit]

True, but honest measurements will tell you what it sounds like. Once you establish your preference, a reliable set of measurements gets you a good understanding of how the speaker performs.

SheepStar

Yes, I expect and welcome subjectivity.

I was reading reviews on a particular speaker and some reviewers says the bass is tight, others say it muddled. There's no telling what these comments are referenced to. Are they even using a set "laboratory" when listening or going my memory? In most reviews it appears they're comparing them to a "reference" system and memory, not directly to other speakers in class. A bad day in the office could equal muddy bass.

I read a car review where they basically said car A, with the most horsepower/weight, eventually put in the fastest lap time at the track. While in the second place car B the drivers got in the car and were able to put in fast consistent laps from the very first first lap. In this case all cars are tested on the same track during the same time. A comment like this, along with comments like this one was more comfortable driving around town for " " reasons can help you choose what's most important to you.

I'd rather be arguing the subtitles between speakers within a class. For me, in the town I'm in and how much free time I have there's no way I'll be able to get all the speakers I'm interested in in my house and check them out at the same time. It would be nice to read that all the reviewer agree this speaker has an edge with this parameter and that on that and I can choose what's important to me. Then I can take two home and compare them.

 

[AcuDefTechGuy]

Yeah, I agree with Seth=L.

I know many of us try to compare speakers and amps to cars, but they are way too different.

There are just so many differences in cars that are obvious.

For example, I recently test drove the Range Rover and Lexus LX570 SUVs. Just starting the engine and listening to the cabin noise was such a drastic difference. And this is just ONE difference. We could compare the smoothness of the ride, the reliability history, safety issues, 4x4 drive vs 2-wheel drive (other cars) in snow, and so many other parameters that could actually significantly affect us. Cars can be a "hobby", but they are a lot more than that.

Speakers and amps are just hobbies. Nothing more. Even if all speakers measure differently and sound differently, audio is still just a hobby and it all comes down to your personal preference.

 

[Seth=L]

True, but honest measurements will tell you what it sounds like. Once you establish your preference, a reliable set of measurements gets you a good understanding of how the speaker performs.

SheepStar

I am not a proponent of measurements being an end all. One can't entirely discern how a speaker will sound based on a few measurements and sometimes even the more comprehensive ones. Even speakers that measure similar you may find they will sound completely different in your home because of how that particular design is going to react with your room.

Now I am also not as learned as I would like to be when it concerns reading comprehensive 3rd party measurements, and I'd wager I'm not the only one who looks at them with befuddlement. As far as I know there aren't any comprehensive measurements that will tell you how exactly a speaker is going to "sound". Is there a measurement that tells you how fatiguing or relaxing a speaker is going to be? Is there a measurement that is going to tell you how deep or wide the soundstage is going to be? Is there a measurement that tells you how big a speaker will sound given it's relative size? There's just so much to consider I can't imagine that there's a better way to compare speakers than to have them in your home.

Before anyone jumps on me for contradicting myself, remember, I buy used speakers almost exclusively. I buy them at the right prices so if I don't like them I can get my money back out of them. I don't like to buy new, especially blind, because I cannot recover my full investment if I end up not liking what I hear.

 

[psbfan9]

A couple of things I've noticed about speaker reviews; Most reviewers use top of the line equipment and external amplification. Why not do a more real world test. Hook the speakers up to a mid level AVR like the majority of readers are using, and use Blue Jeans or Monoprice cables.
From reading speaker reviews, speakers seem to be more similar than different. Speakers are going to sound different from to room.
Do you guys remember when the reviewer said ALL speaker reviews should be positive because it would be good for the industry?

 

[Hobbit]

Yeah, I agree with Seth=L.

I know many of us try to compare speakers and amps to cars, but they are way too different.

There are just so many differences in cars that are obvious.

So are you saying that if a few of us got together with 4 similarly priced speaker in the same category ($1000 towers) we wouldn't be able to say this speaker has the best imaging, this one the biggest soundstage, this one the best bass, and this one the easiest placement, etc? Indeed some may be too close to judge....

Of course, like with cars, we'll have preferences. If I could afford to drive a Ferrari, I wouldn't care if a some Chevy can beat me 0-60 because I'll be in a Ferrari

 

[Swerd]

I find it easy to agree with everyone who's already posted on this thread.

Audio magazines have quite different business models and general cultures than auto magazines. I doubt if the advertisers in audio magazines would permit their speakers to be directly compared to others in a "shootout" format.

Performing such a shootout under blinded conditions that control for individual listeners' attitudes, beliefs, and expectations has not been easy.

Speakers are also sold under very different conditions and with a wide variety of markup prices. The old traditional small brick & mortar shops (nearly all gone) usually sold speakers with at least a 50% markup. Their cost from the distributor was half of the retail price. So even with a "large" discount, speakers were sold by these stores with plenty of profit built-in.

The large box stores (such as Best Buy) specialized in lower prices due to high volume, selling speakers made by the very large companies who made money by the economy of scale.

The Internet Direct speaker companies (at least some of them) offer significantly lower prices and higher sound quality than the old B&M shops by eliminating the distributor markups. It can honestly be claimed and demonstrated that a $2000 pair of ID speakers perform as well as a $4000 pair sold by a B&M shop at retail prices. But you rarely find an ID speaker reviewed by the audio magazines.

In comparison, I find auto dealers to be much more competitive in prices than audio dealers.

 

[Swerd]

Most reviewers use top of the line equipment and external amplification. Why not do a more real world test. Hook the speakers up to a mid level AVR like the majority of readers are using, and use Blue Jeans or Monoprice cables,

This is an example of what I mean by the "culture" of audio magazines. The reviewers all use top of the price range equipment and "exotic" cables because the editors of the magazines depend on the income from advertisers who make and sell that stuff.

Auto magazines that promoted such laughable ideas as the "snake oil" promoted by some audio magazines, would soon fail. The auto buying market is too large, and includes too many different price ranges, to permit such nonsense.

 

[Seth=L]

In comparison, I find auto dealers to be much more competitive in prices than audio dealers.

Absolutely, that market isn't as easily dismissed. Automobiles are considered a necessity in most parts of the world. As DefTechGuy says, speakers are a hobby. Sure, there are auto hobbyists, but home audio is almost exclusively a hobby. Unfortunately the way home audio is now it's directed at people who have a passing interest in something better than a boombox or tv speakers (refer to any of the Dumbing Down of Audio threads).

 

[Swerd]

Absolutely, that market isn't as easily dismissed. Automobiles are considered a necessity in most parts of the world…

A number of years ago, when internet prices first appeared for auto sales, local auto dealers responded quickly with similar prices. They couldn't afford to loose those sales.

Compare that to the B&M audio shops. Their response to internet prices was the opposite. And that is why they are disappearing.

 

[AcuDefTechGuy]

So are you saying that if a few of us got together with 4 similarly priced speaker in the same category ($1000 towers) we wouldn't be able to say this speaker has the best imaging, this one the biggest soundstage, this one the best bass, and this one the easiest placement, etc? Indeed some may be too close to judge....

Of course, like with cars, we'll have preferences. If I could afford to drive a Ferrari, I wouldn't care if a some Chevy can beat me 0-60 because I'll be in a Ferrari

Soundstage, imaging, etc. is cerebral perception and we all may perceive the sensations differently.

Whether one tweeter is sweeter or smoother or less harsh than another is subjective. Klipsch fans don't think Klipsch sound bright, forward, harsh at all.

Definitely Tech fans don't think Def Tech speakers sound muddy in the midrange or bass at all. They think their speakers sound great.

Some people think bi-pole and di-pole speakers have a larger soundstage, some people don't think so at all.

If the magnitude of the bass is drastically different, then that could be easily perceived (one can play 30Hz and one can only play to 60Hz).

 

[lsiberian]

Speakers have too many variables to be compared solely on measurements. For example my bipolar speakers are going to measure vastly different from a traditional speaker. To select the best speaker based on measurements you need to compare the optimal acceptable placement of each speaker in the space vs every other speakers in the comparison. The effort required to do this is so cost prohibitive that it's unpractical for any reviewer to take on. On tap of that we all tolerate different short coming in our speakers differently. We also enjoy different genres of music. If my favorite CD was Hootie greatest hits I would be a fool to use a setup like TLS_Guy's. However if my favorite CDs were well mastered Mahler symphonies then I would love his setup.
One of the primary reasons I love building my own speakers is that I can tailor the design to please my own tastes instead of someone else's. At the end of the day none of us like the same Spaghetti Sauce or Ice Cream. Even in ABX testing at Harman Studios there were variances in people's favorite speakers. These are trained experts disagreeing on the subject. This is why you need to listen to speakers yourself and yes you can actually get an idea for what you prefer at the local botique shop. Listen to horns and compare them to soft domes. Listen to bass heavy towers and solid bookshelves. At the end of the day you'll find the type of speaker you love most. For me I love a clean sealed bipolar sound. Others will prefer the Boom of a ported design.

 

[GranteedEV]

I'll bet most $1000 will end up performing more similar than different.

There are so many different ways to skin a proverbial cat - different target presentations, different ways to cut cost corners, and different designer assumptions. Potential buyers don't know what they want or if they even want it, so the manufacturers take liberty and make that call themselves.

 

[AcuDefTechGuy]

Automobiles are considered a necessity in most parts of the world.

There is no way I'm walking, especially in 10F or 100F degrees.

 

[Cos]

I am not a proponent of measurements being an end all. One can't entirely discern how a speaker will sound based on a few measurements and sometimes even the more comprehensive ones. Even speakers that measure similar you may find they will sound completely different in your home because of how that particular design is going to react with your room.

Now I am also not as learned as I would like to be when it concerns reading comprehensive 3rd party measurements, and I'd wager I'm not the only one who looks at them with befuddlement. As far as I know there aren't any comprehensive measurements that will tell you how exactly a speaker is going to "sound". Is there a measurement that tells you how fatiguing or relaxing a speaker is going to be? Is there a measurement that is going to tell you how deep or wide the soundstage is going to be? Is there a measurement that tells you how big a speaker will sound given it's relative size? There's just so much to consider I can't imagine that there's a better way to compare speakers than to have them in your home.

Before anyone jumps on me for contradicting myself, remember, I buy used speakers almost exclusively. I buy them at the right prices so if I don't like them I can get my money back out of them. I don't like to buy new, especially blind, because I cannot recover my full investment if I end up not liking what I hear.

I agree with your comments here. I do not want to discount measurements, but as have many others have said, when you talk imaging, clarity, detail, it can be subjective. I am not going to worry about a speaker that might be harsh at over 16hz because I can't hear it. Speaker testing is done an Anechoic chamber. I would wager that 99.9999% of people don't have this type of setup for the 2 channel listening area or home theater, so sound can get affected by many different criteria.

I have heard speakers described as forward, etc. and I am not educated enough to look at a graph to determine what in the measurements tell me if a speaker is going to sound forward. Measurements may also be effected by what equipment you use, etc.

To use the car analogy. Car specs, rated horsepower, 0-60 lap times etc are all great in a controlled environment. But I do not have similar conditions on my commute to the office or expressway, nor will I ever be able to match a professional driver on a track. Its a point of reference that is great to have as a resource or foundation, but is affected greatly by real world environments and situations.

 

[Hobbit]

I'm finding it hard to believe the 4 expert reviewers can't come into a typical living room, set up speaker A, and then speaker B, and not be able to at least agree that one has better bass and one has better imaging etc. Beyond that I'm sure they'll have their personal preferences to which speaker they would buy based on the entire package. Most of us realize there will always be flaws in any review.

Does that also mean that guys that offer HT setup services are spreading snake oil around the room? I have friends who swear by these services.

This is not that different than a car. Like my example, if you're an expert driver doing autocross you may prefer the fastest car from that group because you can manhandle it for a given lap. The rest of us may prefer the 2nd place car that was easier to drive. Or, you may prefer the most comfortable one because you don't plan on racing. None of these are directly related to published specs (the most HP doesn't always mean faster lap times, just watch F1). I didn't buy my slk320 on specs alone.

 

[AcuDefTechGuy]

I'm finding it hard to believe the 4 expert reviewers can't come into a typical living room, set up speaker A, and then speaker B, and not be able to at least agree that one has better bass and one has better imaging etc.

If a car can't handle the 50MPH around the same curve, it would fly off the road.

If a car can't stop fast enough, it would hit the object.

A 2-wheel drive car will have less traction and slide a lot more on snowy/wet roads than a full-time 4-wheel drive.

It is black and white. There is nothing subjective about that.

Speaker measurements may be objective and may be compared to car measurements.

But how well you like the bass or imaging or soundstage of a speaker is still 100% subjective.

 

[Sheep]

I am not a proponent of measurements being an end all. One can't entirely discern how a speaker will sound based on a few measurements and sometimes even the more comprehensive ones. Even speakers that measure similar you may find they will sound completely different in your home because of how that particular design is going to react with your room.

Now I am also not as learned as I would like to be when it concerns reading comprehensive 3rd party measurements, and I'd wager I'm not the only one who looks at them with befuddlement. As far as I know there aren't any comprehensive measurements that will tell you how exactly a speaker is going to "sound". Is there a measurement that tells you how fatiguing or relaxing a speaker is going to be? Is there a measurement that is going to tell you how deep or wide the soundstage is going to be? Is there a measurement that tells you how big a speaker will sound given it's relative size? There's just so much to consider I can't imagine that there's a better way to compare speakers than to have them in your home.

Before anyone jumps on me for contradicting myself, remember, I buy used speakers almost exclusively. I buy them at the right prices so if I don't like them I can get my money back out of them. I don't like to buy new, especially blind, because I cannot recover my full investment if I end up not liking what I hear.

I said a good understanding, not the be all end all. I dislike harsh or forward mid and high end. If I see a speaker with lots of extra gain in the treble, I can usually assume that I'm not going to like it. I will still listen to it and take the room and setup into acount, but I know this preference and have been able to spot it before.

SheepStar