Subjective Loudspeaker Reviews - All About Price?

A

admin

Audioholics Robot
Staff member
When taking on the task of reviewing products for consumers there are different ways to get across the conclusions and experiences of the reviewer. Objective testing and subjective viewpoints are both presented, and price is also included in the evaluation process. But there are so many methodologies and considerations, how does Audioholics keep it all straight? Frank asks a very valid question and we thought it was good enough to include it for public consumption to outline just how Audioholics handles this issue.


Discuss "Subjective Loudspeaker Reviews - All About Price?" here. Read the article.
 
3db

3db

Audioholic Overlord
When taking on the task of reviewing products for consumers there are different ways to get across the conclusions and experiences of the reviewer. Objective testing and subjective viewpoints are both presented, and price is also included in the evaluation process. But there are so many methodologies and considerations, how does Audioholics keep it all straight? Frank asks a very valid question and we thought it was good enough to include it for public consumption to outline just how Audioholics handles this issue.


Discuss "Subjective Loudspeaker Reviews - All About Price?" here. Read the article.

I can't think of any review that would not include price into the equation. Its the only way I know of that forces the reviewers to compare apples to apples and not apple to oranges. What I mean is , I would think it unfair to compare a speaker in the $500 range against a speaker in the $2500 range and expect the cheaper speaker to score decent marks. Yes I know there are exceptions but I'm speaking in terms of generality here. Your scale would become too large and meaningless if you tried to score on an absolute basis.
I can point to a review done by Audioholics where the reviewer gave a very unflattering review to PSB Alpha series and it was unflattering not becuase they didn't do well in their price bracket, they didn't do well compared to speakers costing 5 times their amount. That to me is the perfect example as to why one cannot use an absolute scale in reviews

This cost association also boils down to the consumer. When people buy speakers, they usually don't go around comparing $500 to $2500 speakers. They stick to the budget as close as possible and compare speakers in similar price brackets.

Subjective comparisons I take with a grain od salt as no one has my hearing nor do I have their hearing. Another man's meat is another man's poison. Subjective comparisons may establish a baseline from which to start but its only just that, a starting point. An audition will quickly realign the starting point of comparison by the cutomer.

Let the chips fall where they may!! :)
 
TLS Guy

TLS Guy

Audioholic Slumlord
Speaker evaluation the toughest of all!

I have to agree with Admin. on this speaker testing issue. Unfortunately with any electromechanical advice pure testing alone is not going to tell you how it sounds. In this aspect speakers are the worst of all. It is well nigh impossible to convey how a speaker sounds in words.

The fact is designing and building a speaker is that tricky combination of science, art and judgment.

In this highly technical world in which we live this tends to get people out of their comfort zone. As a critical care physician this is well within my comfort zone!

If you go to the B & W site you will see that they are very honest on this point, they do good design and measurement, but have to voice even their most expensive speakers after extensive listening tests.

I have been building speakers for over 50 years, and I can tell you that is true. Worse than that it takes several months of listening to get to know a speaker. The late John Wright of TDL, who I think produced some of the finest speakers of his generation, taught me that. Once he had a speaker good, he would not tinker with it until three or four months of listening had elapsed. This is how you progress from good to excellent.

I understand the members' problem. They want to buy speakers at heavy discount over the Internet, sight unseen. I'm sorry, but at the current state of the art it is just not possible to build an excellent system this way.

So the golden rules are.

Don't buy a speaker you have not heard.

Even if you have heard it, make sure you can return it at least within thirty days. Be warned however that even 30 days may not be enough for the speaker to reveal all it's vices to you.

When it comes to speakers there really is no substitute for a competent, understanding and accommodating local dealer. Unfortunately they are a dying breed, the big box and Internet retailers have done it for most of them.

Actually that allows unscrupulous speaker manufacturers to get away with murder, of the music anyway.
 
Seth=L

Seth=L

Audioholic Overlord
Consumer reports only has 6 speaker companies that they review? That is stupid.
 
annunaki

annunaki

Moderator
I could see matching up speakers using a more objective scale with a price barrier.

For instance:

MAP pricing $45-$250 has a set of parameters to meet or a reference standard to which others are judged, $500-$750 (MAP) another, $750-$1,250 (MAP) another set, $1,250-$2,500 (MAP), and $2,500+ (MAP). You could associate a reference standard and expected performance envelopes in each of the 5 categories. Comparing speakers from one price point to another would be easier and give a general idea of what may be expected by moving up in price each time. It would definitely let a person know if the speaker does better or worse than the norm for the group.

Sure it is not perfect but no system will ever be as someone will always make a speaker that falls on the fringes of one category or another.

The biggest thing to stress with an outlined semi-objective system would be to listen before you buy and/or make sure a speaker has a good return policy. How it sounds to the purchaser is what matters most.
 
skizzerflake

skizzerflake

Audioholic Field Marshall
The other 800 pound gorilla in the situation is that even if there is a measurement based, objective measure of a speaker's accuracy, when you take it home, a speaker becomes part of a room that is different from the dealer's room, the reviewers room and every other room in your house. The dimensions, materials, carpeting, furniture, placement and everything else changes the way the speakers sound. Not only do you need to never buy a speaker you have not heard, but even then, it will be an adventure until you hear it in your own situation.
 
TLS Guy

TLS Guy

Audioholic Slumlord
The other 800 pound gorilla in the situation is that even if there is a measurement based, objective measure of a speaker's accuracy, when you take it home, a speaker becomes part of a room that is different from the dealer's room, the reviewers room and every other room in your house. The dimensions, materials, carpeting, furniture, placement and everything else changes the way the speakers sound. Not only do you need to never buy a speaker you have not heard, but even then, it will be an adventure until you hear it in your own situation.
It is true that the room does affect the perceived sound to a degree, and with far too many speakers to a high degree. A speaker that sounds very different in different environments has something the matter with it. I have built speakers I have taken them on location recording in a lot of venues. A significantly different sound in different locals is a red flag for trouble with a speaker.

The reason is that the off axis response is reflected back to the listener within the Haas fusion zone. The take home lesson is that the off axis response must closely mirror the axis response. The axis HF response can roll off, but the mid band must mirror the axis response closely. If not the reflected response will clearly color the sound. This will result in a speaker being fussy as to what room it is in.

I know this will start another controversy, but I believe a lot of this room treatment is to provide a balm to speakers with a poor waterfall response graph.

Honestly my speakers do not need room treatment, nor do other superior loudspeakers.

I have been asked in other threads why there is no room treatment in this space.

http://mdcarter.smugmug.com/gallery/2424008#127077317

Well this speaker system recreates acoustic spaces that I'm familiar with, with uncanny accuracy and I just don't need to spoil the aesthetic plastering room treatments about.

I've spoken to other engineers that use very good location monitors, and they agree with me that good speakers are not fussy.
 
mtrycrafts

mtrycrafts

Audioholic Slumlord
Consumer reports only has 6 speaker companies that they review? That is stupid.
CR is not published to please the audio crowd but their subscribing public who are more in the 'general' category and what may be readily available to its readers. That is my take on it, but perhaps a letter to them may shed a different light on the subject.:D
 
jaxvon

jaxvon

Audioholic Ninja
Honestly my speakers do not need room treatment, nor do other superior loudspeakers.
In what universe do your speakers exist? That is a seriously arrogant and ignorant statement about the scientific reality of sound reproduction. You cannot make "room proof" speakers. However, I will let you sit and enjoy your inferior sound.
 
WmAx

WmAx

Audioholic Samurai
Honestly my speakers do not need room treatment, nor do other superior loudspeakers.
It may be true that your speakers don't need room treatment in order to satisfy your personal standard(s) for audio quality.

However, where higher quality standards are required, there is no avoiding proper room acoustic(s) behaviour, which in real rooms, requires physical treatements/correction(s).

A speaker with a substantially poorer off axis response will indeed sound worse in a untreated room vs. one with a linear off axis reponse. This is concluded in perceptual studies by Toole. But in no case is a speaker optimal with no room treatments. A speaker with even off axis response can be optimized with less treatments; but a substantial amount are still required to optimize sound quality.

-Chris
 
AcuDefTechGuy

AcuDefTechGuy

Audioholic Jedi
It may be true that your speakers don't need room treatment in order to satisfy your personal standard(s) for audio quality.

However, where higher quality standards are required, there is no avoiding proper room acoustic(s) behaviour, which in real rooms, requires physical treatements/correction(s).

A speaker with a substantially poorer off axis response will indeed sound worse in a untreated room vs. one with a linear off axis reponse. This is concluded in perceptual studies by Toole. But in no case is a speaker optimal with no room treatments. A speaker with even off axis response can be optimized with less treatments; but a substantial amount are still required to optimize sound quality.

-Chris
Shoot.

What's to lose but a few hundred bucks?

Even I bought acoustic panels!:D

I can't say that I can actually hear a significant difference, but I'm sure those 2' x 4' x 4" panels are doing their job. Can't take any chances.

Like you said, some speakers/room need less treatments than others.

Perhaps TLS Guy's system needs LESS treatment.
 
WmAx

WmAx

Audioholic Samurai
Shoot.

I can't say that I can actually hear a significant difference, but I'm sure those 2' x 4' x 4" panels are doing their job. Can't take any chances.

Like you said, some speakers/room need less treatments than others.

Perhaps TLS Guy's system needs LESS treatment.
And who's to say you used enough of them, or placed them for optimal effect? :)

-Chris
 
J

JackVa1

Junior Audioholic
CU and speakers

I have not read a CU report on speakers in years. Back in the 70's they used to report accuracy scores. They would then recommend a few low, mid, and high end system combinations and the reader could mix and match receivers, speakers, tape decks.

Each speaker pair did have a score though and I wish I could get the old issues and read how it was measured. CU was pretty much rating how well a speaker took a source and accurately reproduced it. CU did mention that it was important for a speaker to not ADD or remove sound. They seemed to have gear to measure with.

Were they all stupid? When I saw speakers with a 94 accuracy score (by CU standard) for $400.00 , I saw no reason to go buy a pair for $1200. So I took my receiver into a store and blind auditioned (A/B) a few of their recomendations and bought speakers.

The speakers CU recommended back then out performed a $1200 pair with my receiver with a few different "records". We still own them and they are what I call "clean" - Low efficiency and they let my receiver do the work - they now are performing for one of my kids and he loves them.

So I say, just give me the scores. I can then go listen.

What's changed?
 
WmAx

WmAx

Audioholic Samurai
I have not read a CU report on speakers in years. Back in the 70's they used to report accuracy scores. They would then recommend a few low, mid, and high end system combinations and the reader could mix and match receivers, speakers, tape decks.

Each speaker pair did have a score though and I wish I could get the old issues and read how it was measured. CU was pretty much rating how well a speaker took a source and accurately reproduced it. CU did mention that it was important for a speaker to not ADD or remove sound. They seemed to have gear to measure with.

Were they all stupid? When I saw speakers with a 94 accuracy score (by CU standard) for $400.00 , I saw no reason to go buy a pair for $1200. So I took my receiver into a store and blind auditioned (A/B) a few of their recomendations and bought speakers.

The speakers CU recommended back then out performed a $1200 pair with my receiver with a few different "records". We still own them and they are what I call "clean" - Low efficiency and they let my receiver do the work - they now are performing for one of my kids and he loves them.

So I say, just give me the scores. I can then go listen.

What's changed?
I don't remember exactly how CU tests speakers. But I do remember Floyd Toole (the most accomplished perceptual researcher on the topic of loudspeakers) or Sean Olive (works in direct conjunction with Toole) commenting on CU in an article and stating that they used insufficient evaluation methods to accurately determine speaker sound quality. Unfortunately, I did not note the article reference source(I believe it was a technical or engineering journal).

-Chris
 
J

JackVa1

Junior Audioholic
I'm a novice, but I can read scores, close my eyes and determine what I enjoy. Do the gentlemen you reference "score" speakers?

I wonder if it's that complicated to capture sound and see if any spectrum is missing.
 
no. 5

no. 5

Audioholic Field Marshall
I don't remember exactly how CU tests speakers.
From what I could gather from this, CR based accuracy on frequency response measurements between 110Hz and 14kHz (presumably measured with a 1/3 octave spectrum analyzer).
 
WmAx

WmAx

Audioholic Samurai
I'm a novice, but I can read scores, close my eyes and determine what I enjoy. Do the gentlemen you reference "score" speakers?

I wonder if it's that complicated to capture sound and see if any spectrum is missing.
It's very complicated, as no single measurement gives sufficient data. Hundreds of measured data points(spanning a variety of test types) is required to begin to be able to conclude a speaker's sound quality - and then if you have this data - it takes an in depth knowledge of the perceptual research in order to be able to correlate measured data with expected human subject reaction(s) in order to come to an informed conclusion. I highly doubt CR even has an expert on staff that could do this properly. Most manufactures, judging from their flag ship speaker designs, do not seem to even have total capability.

-Chris
 
majorloser

majorloser

Moderator
A speaker that sounds very different in different environments has something the matter with it. I have built speakers I have taken them on location recording in a lot of venues. A significantly different sound in different locals is a red flag for trouble with a speaker.
Unless the room is an anechoic chamber it can always benefit from some sort of room treatment or even interior design considerations. Without some reflection and absorbtion a room will have no "ambiance". The sound will never achieve the level of a live performance. Even your Haas Fusion principle takes into account that perfectly reflected sounds can merge into one apparent source to the listener.

I'm sure everybody has been in a room that has reflections so extreme that even normal conversation creates loud echoes. I can think of some rather large round concrete tanks I've been inside.
 
J

JackVa1

Junior Audioholic
AVID 102's

From what I could gather from this, CR based accuracy on frequency response measurements between 110Hz and 14kHz (presumably measured with a 1/3 octave spectrum analyzer).
Thank you ! I totally enjoyed reading the CR from 1976.

The AVID's were the speakers I bought and now passed to my son. Trust me, they were and are still good speakers no matter what you think of CR. I swear I blind tested them agains a pair of large ADVENTS, a weird speaker that looked like a large white screen, and a big speaker that sat in a corner with the midrange or tweaters actually aimed into the corner of the room (these suckers weighed a ton).
All were pricy.
 
davidtwotrees

davidtwotrees

Audioholic General
I used to be a fairly serious wine drinker. The Wine Spectator had a one hundred point scale for any bottle of wine it reviewed. While it wasn't perfect, the numbers said some things pretty clearly to me a wine drinker.
A bottle of wine scoring in the eighties was a solid performer, one scoring in the nineties was something special, and a bottle that scored 98, 99, or the rare 100 was an exceptional bottle of wine that would knock your socks off.

If you start to include price in there, a bottle that scored 85 and cost $400 was a terrible value, while one that scored 85 and cost $12 was an exceptional value.

So, I disagree wholeheartedly that speakers cannot be scored. Yes, there is no perfect review methodology, and there are many variables. But as a consumer, I want the expert to give me his best judgement on a speaker's performance. Then tell me what it cost, and I as an informed consumer can make my personal value determination.
 
newsletter

  • RBHsound.com
  • BlueJeansCable.com
  • SVS Sound Subwoofers
  • Experience the Martin Logan Montis
Top