They may not here, but they do endlessly elsewhere and I'd like to actually develop a protocol to test it to see if I/we can get them to STFU about it some day. I also simply find it an intellectual/science discussion on how/if it can indeed be tested per my OP.
OK, I tried listening to that YT video. It was too long. By minute 12, they still hadn't made their point, and I gave up. But I heard enough to have a couple of simple observations:
They used two speakers in stereo. I would simplify things and use only one speaker in mono, without a subwoofer.
They made the mistake of asking too much of listeners: which cable did they prefer? Instead, they should have asked only if they could hear a difference. Avoid preferences, a subjective matter. Design the test so all answers are a simple YES or NO.
They did recognize that they didn’t test enough people. Apparently, someone told them they needed to test 100 people to achieve statistical significance. Maybe and maybe not. A test of roughly 30 to 100 people can really only answer the very simple YES or NO question “Is it worthwhile to test a larger group, 300 to 1000 people?”
The question becomes, how many people must be tested to provide an overall YES answer with a ≥95% Confidence Interval (95% CI). If the individual YES answers are close to the NO answers, such as 55% vs. 45%, more people are needed to achieve that 95% CI. If the difference between YES and NO is large, you can get away with fewer people. To get this right, one needs to really know statistics, or have a friend who does. (By confidence interval, I mean how many individuals must be sampled from all humans to provide an answer that we are at least 95% confident is not due to statistical sampling errors.) If I didn't warn you that statistics is boring, I'm telling you now.
The purpose of this listening test is to demonstrate whether listeners can actually hear differences in a speaker's sound due to different speaker cables. Statistics, properly used, is essential for this. So are properly designed controls.
Experimental Controls
This listening test is really an experimental effort to measure something. It’s not an established test. How good is the test? How sensitive is it? What is the background noise level? How reliable is the test? In science experiments, these questions must always be answered at the same time as the main experiment is done. They're answered with the experimental controls. With good controls, strong conclusions can be made. With so-so controls, some conclusions can be made, but they’ll be weaker. Without controls, the main experiment is essentially meaningless. When all the audio subjectivists start attacking you because they don’t believe your results, you'll wish you had done better controls.
Negative Controls
Each listener must be tested when two identical cables are compared. In theory, they should all answer “NO, they’re not different”. But you cannot assume people can always tell when cables are identical. You can think of a Negative Control as a measure how many “false positive” answers there are. You must measure the amount of these false positive answers that each listener reports. The false positive percentage for each listener must be subtracted from the percentage of YES or NO answers that listener provides when there really were two different cables being tested. If all listeners have a low average false positive rate, you would be in good position to make useful conclusions about the cable comparison results. If that average false positive rate is high, 50% for example, you could only conclude that listeners could not reliably report what they heard. No useful conclusions about the cable comparisons could be made.
This isn't about dishonesty. It's that no one is infallible. You can think of the Negative Control as a measure of the background noise in the test. It won’t be zero. Measure it and find out what it is.
Positive Controls
Each listener must be tested when different levels of pink or white noise is added on top of the sounds or brief music passage used during a listening test. These tests are completely independent of speaker cables, but they are needed as comparisons to the speaker cable tests. Imagine a series of listening tests where each listener hears 0% added noise vs. a series, such as 0.3%, 1%, 3%, 10%, or more. Imagine a graph of these results, where the X axis shows added noise from 0% to 10%, and the Y axis shows how many listeners could reliably answer "YES, I can hear it". With such a standard curve, you could measure what level of added noise could be heard by 50% of the listeners. This might be useful as a comparison to what percentage of listeners could hear differences, if any, between speaker cables.
The positive controls measure what genuine levels of noise or distortion each listener actually can reliably detect. But I’m just guessing what these noise levels should be, for discussion’s sake. What range of added noise actually works will have to be determined ahead of time, before doing the real cable comparison listening test.
Once you have that result, you can compare the Positive Control results to the cable comparison results. The goal is to be able to say something like this: “Under test conditions, where 50% of the listeners could reliably hear 2.3% added noise (a made-up value), 35% of listeners (another made-up number) could detect a difference between speaker cables. It helps put things into context. If other people try to repeat this test, the Positive Controls can tell you if their test was more, less, or similarly sensitive.
As you can imagine, good Positive Controls aren’t as easy to come up with or perform as Negative Controls. But you must run appropriate Positive Controls to have an idea how sensitive the whole cable comparison test actually is.
Again, this isn’t about dishonesty. The sounds due to different cables – if they exist at all – might be very subtle. No one can always hear differences no matter how subtle they might be. Measure it and find out.
If you imagine how many individual listening tests each listener must endure, you'll have an idea how long this will take. It won’t be easy. And, all this testing is much more interesting when there is likely to be a real difference. Listening to different speaker cables won’t provide that kind of excitement. Unless you think listening to paint dry is exciting.
