Setting up a Speaker Shootout or Component Comparison the Right Way

[admin]

In preparation for the forthcoming speaker shootout, we thought it'd be a good idea to explore the correct way to set one up. One of the most popular articles we ever write on Audioholics is speaker or cable shootouts. Shootouts are really just a comparison of two or more products. This sounds like a fairly straightforward process where you place two competing products in the same room and take a listen/look. But the reality is that it is much more complicated.


Discuss "Setting up a Speaker Shootout or Component Comparison the Right Way" here. Read the article.

[highfigh]

Is the test designed to compare each speaker in its recommended position? I know some sound best against a wall, fairly far away and somewhere in-between. Is the room treated?

[Djizasse]

Nice, but I was really hoping for a more in depth guide. Where's the Part II?

[Loren42]

Everything you do is worthless unless you use a double blind testing format.

By that I mean both the test subject that is judging the comparison and the person conducting the test are absolutely unaware of which item is being tested when. Results are tallied by an independent researcher.

Then you need to repeat the test enough times to be sure any results are statistically relevant.

The problem with the above method is that it will rapidly show, for instance, that speaker cable is meaningless. The blind test that used Monster Cable and coat hangers is a great example of this. Once you go on to prove that coat hanger sounds the same as expensive speaker wire your advertising revenue will drop to zero.

So my bet is that you won't do a proper test.

[gene]

--------------------------------------------------------------------------------

Everything you do is worthless unless you use a double blind testing format.

By that I mean both the test subject that is judging the comparison and the person conducting the test are absolutely unaware of which item is being tested when. Results are tallied by an independent researcher.

Then you need to repeat the test enough times to be sure any results are statistically relevant.

The problem with the above method is that it will rapidly show, for instance, that speaker cable is meaningless. The blind test that used Monster Cable and coat hangers is a great example of this. Once you go on to prove that coat hanger sounds the same as expensive speaker wire your advertising revenue will drop to zero.

So my bet is that you won't do a proper test.

Um I strongly disagree that it has to be DBT. If the listeners are unaware of the speakers they are listening to they can still make relatively excellent unbiased judgements. I often do quick comparisons of speaker pairs when doing reviews without any DBT or ABT testing and its easy to hear clear differences between speakers as long as they are properly level matched and positions (ie. L1L1 R1R2).

Cables are another story b/c the sonic differences are so subtle and often non existant, something we always stress. Speakers often sound drastically different which makes it much easier to setup a fairly accurate test without doing a true DBT.

[majorloser]

"In preparation for the forthcoming speaker shootout...

Okay, I'll bite. What ya' got planned?

[R-Carpenter]

Speakers are more difficult to compare due to on and off axis response (crossover, driver differences and combination) and room interactions. You really need to place compared speakers in the same position for a DBT. Inexpensive SPL meter and some pink noise will make level matching easy enough. Large screen with acoustic fabric will make it truly double blind.
Sometimes a A-B speaker switch box cam make even easier job.
As far as Blind or not blind, I disagree. Nice looking speakers usually admired first for the looks and automatically regarded as good sounding.
Perhaps a professional reviewer has a less biased view but your average customer will buy with their eyes.

[3db]

Speakers are more difficult to compare due to on and off axis response (crossover, driver differences and combination) and room interactions. You really need to place compared speakers in the same position for a DBT. Inexpensive SPL meter and some pink noise will make level matching easy enough. Large screen with acoustic fabric will make it truly double blind.
Sometimes a A-B speaker switch box cam make even easier job.
As far as Blind or not blind, I disagree. Nice looking speakers usually admired first for the looks and automatically regarded as good sounding.
Perhaps a professional reviewer has a less biased view but your average customer will buy with their eyes.

I tend to agree with that assessment. Most people will buy an aesthetically prettier speaker whose sound is marginaly worse than the ugly duckling beside whose sound is marginaly better.

[BoB/335]

I tend to agree with that assessment. Most people will buy an aesthetically prettier speaker whose sound is marginaly worse than the ugly duckling beside whose sound is marginaly better.

I agree! The Song Towers I listened to side by side with M80's were much better looking so that's what I'm going with.

[Hipnotic4]

When testing speakers, I always found it was better to test with not just familiar material, but new material as well...the new material helps my ears pay more attention to what sounds or should sound "right"...

Looking foward to the shootout though!