I think there's some confusion here about what blind testing is and when it should be used.
For the simple purpose of comparing two different amplifiers, a simple sighted test is good enough. As long as the switching between devices is rapid, and the volume levels are matched closely, sighted testing can readily answer the question whether one sounds different or if one is preferred over the other. It will be simple, as long as there really is a noticeable difference in sound. And (this is important) it will be good enough if you don't intend to make the results public.
If two different amplifiers sound similar enough that people can't readily hear any difference, and
if you want make the results public, you need to do blind testing and to pay some attention to statistics. If you ignore this, and publish the results, you are setting yourself up for intense criticism from all sides in the great debate. If enough tests are done to satisfy standard statistics, it will possible to say that listeners could not hear differences. This is a more difficult conclusion to make than when people clearly hear differences.
A single blind test (SBT) is good enough for this purpose. Double blind tests (DBT) are rarely ever required in the world of medical clinical trials, and they are certainly not twice as good as a SBT
. Imagine a clinical trial comparing an experimental drug to a placebo. In a SBT, the patients would not know if they are getting the trial drug or a placebo, but the medical staff do know. And in a DBT the patients
and the medical staff are kept from knowing. If the trial requires that medical staff see the patients on a regular basis for a longer period of time, over several weeks or longer, then there might be a good reason for a DBT over a SBT. I can't see how this would apply to listening tests of amplifiers or speakers, which will take place in one day. Single blind testing should be good enough.
Statistics. Ugh! This is a subject worth entire academic departments, and I can't possibly deal with it adequately here. To decide how many repetitions of a listening test are enough, you must have an idea (from prior testing) just what percentage of listeners might hear a difference when there is a real difference. If two amps sound so similar that roughly 50% of listeners say they can hear a difference, you have to also ask how many hear a difference when both amps are identical. If that is also 50%, you will have to raise the bar pretty high before you can conclude that people really can hear differences. Once have an idea about that, then you need to look up statistics books and read all about what 95% confidence intervals mean
. Then we can talk about how many trial repetitions are needed to make valid conclusions.
I could go on and on about this (I have probably lost readers by now
). But I want to make it clear that I agree with Gene that simple sighted tests are appealingly simple, and easy to do when differences are easy to hear. But if you want to convince the doubters in this world, you better pay attention to blind testing and statistical significance.
