How to Skew a Blind Listening Test

[admin]

Setting up a blind test is easy right? Or is it? While most people will take the most well known precautions, they often forget or don't know about some of the others than can seriously affect the outcome of the test. Here are some lesser known things to remember when you set up your listening test.


Discuss "How to Skew a Blind Listening Test" here. Read the article.

[STRONGBADF1]

Good article, Tom. I would also like to see someone do research to see if certain sound signatures do better in quick tests. I would think speakers with more bass energy would fair better in quick tests and that might be tiring to some after living with it.

[fredk]

Excellent article. Even if you don't do proper blind listening when selecting speakers, I think its helpful to understand how you can become biased

[jostenmeat]

I saw someone recently quote Dennis Erskine, on a "skewed", and arguably "malicious" test. Not blind, but it still has a lot of relevance to the second paragraph, Seeing is Believing: "Obviously, the most harmful thing for a participant to have in a listening test is preconceptions."

There are two camps and always will be ... real or imagined. I tested this early on with a recording that (1) was recorded specifically for multi-channel; and, (2) both the DD and DTS version were from the same master, not remixed, levels not changed, etc.

I set the surround processor to display DTS when DD was playing and it displayed DD when DTS was playing. After the listening session 100% of those professing DTS was superior, still agreed DTS was superior. 100% of those who thought DD was superior still were in the DD camp at the end of the session.

Oh well.

[Brewed]

Is it not ludicrous to think that any "blind" test of speakers is tainted?

Surely only one pair of the test candidates can set up properly from an imaging point of view. And in this case, it would surely be Paradigm that dialed in their position and flopped the others where they may lay?

Tom, you know this. I've read your stuff.

Any speaker, regardless of price, has an optimal location in any given room. If you get a nicely centered vocal image but the sound is too bright, you've bought the wrong speaker for your taste. And vice versa.

And did I mention the sympathetic vibrations of the speakers not in use during a "blind" test?

My two cents......

[gene]

I also firmly believe by placing multiple pairs of speakers side by side, you can cause excessive diffraction from adjacent baffles which could hamper the better speakers ability to image. I personally don't like to compare more than two pairs of speakers at a time for this reason and also b/c positioning relative to the listener becomes too drastically different.

In addition, some speakers require more/less toe-in to perform optimally. When placing speakers side by side, you typically nullify this by placing all of the speakers with no toe-in at all.

I think a combination of blind and sighted tests (blind first, sighted after) over a course of a few days is prudent in determining which speaker you really like better, assuming the pairs in question are close enough in sound quality. Music selection is also key as I've seen too many tests done with highly compressed rock music, thus diminishing the finer qualities of the better performing speaker.

One must also take in account to listening level. How linear is the speaker at low and high volume? How consistant does it sound over the SPL range you normally listen too?

If you really want to get scientific in determining which speaker is "better" you need a very large panel of listeners to get a good/accurate statistical sampling. As Tom points out, these listeners should be neutral people not working for the company whose products are under test.

Great article Tom!

[GirgleMirt]

++!

Well done!

A switcher could also be used which could degrade the performance of certain speakers (Using resistors or L-pad for SPL matching of multiple speakers)

And not all speaker's ideal location are the same. Some might be better off closer to the rear wall, closer together, etc...

[gene]

A switcher could also be used which could degrade the performance of certain speakers (http://forums.audioholics.com/forums...ad.php?t=58393)

And not all speaker's ideal location are the same. Some might be better off closer to the rear wall, closer together, etc...

good point. Years ago when I went to Sound Advice to audition speakers, they had them fed thru a switcher. At the time they always seemed to promote B&W over any other brand they carried. When I listened between a pair of B&Ws and Kefs, I preferred the former. Then I asked to remove the switcher and feed the speakers directly to two identical amps, level matched. The listening results were much different then. The salesman was also shocked and claimed the switcher must be defective. After that experience, I was careful to examine all variables when auditioning under someone elses test conditions.

[STRONGBADF1]

I also firmly believe by placing multiple pairs of speakers side by side, you can cause excessive diffraction from adjacent baffles which could hamper the better speakers ability to image. I personally don't like to compare more than two pairs of speakers at a time for this reason and also b/c positioning relative to the listener becomes too drastically different.

In addition, some speakers require more/less toe-in to perform optimally. When placing speakers side by side, you typically nullify this by placing all of the speakers with no toe-in at all.

I think a combination of blind and sighted tests (blind first, sighted after) over a course of a few days is prudent in determining which speaker you really like better, assuming the pairs in question are close enough in sound quality. Music selection is also key as I've seen too many tests done with highly compressed rock music, thus diminishing the finer qualities of the better performing speaker.

One must also take in account to listening level. How linear is the speaker at low and high volume? How consistant does it sound over the SPL range you normally listen too?

If you really want to get scientific in determining which speaker is "better" you need a very large panel of listeners to get a good/accurate statistical sampling. As Tom points out, these listeners should be neutral people not working for the company whose products are under test.

Great article Tom!

I believe the NRC (or maybe it was Harmon) uses a "turntable" to quickly rotate the speakers in and out. Still doesn't help with optimum positioning (unless its huge) but it's got to help some.

[gene]

I believe the NRC (or maybe it was Harmon) uses a "turntable" to quickly rotate the speakers in and out. Still doesn't help with optimum positioning (unless its huge) but it's got to help some.

Yes I've seen that but I wonder what the switch out time was during that process.