Can Objective Loudspeaker Measurements Predict Subjective Preferences?

[gene]

Accurately measuring loudspeaker performance is pretty much a universal problem among most loudspeaker manufacturers and ALL AV review publications. Harman on the other hand under the guidance of Dr. Floyd Toole and Sean Olive have developed a purely scientific methodology that actually allows them to objectively measure and predict listener preference. This article explores the measurements while also providing the secret formula of the characteristics necessary for producing loudspeakers that score highly in subjective listening tests. This is a must read with a call to action to your favorite loudspeaker company to give us more useful information about their products.

Update (6/11/15): I added a poll question so please cast your vote and give us a reason behind your choice.

Read: How To Objectively Measure Loudspeaker Performance

 

[3db]

Accurately measuring loudspeaker performance is pretty much a universal problem among most loudspeaker manufacturers and ALL AV review publications. Harman on the other hand under the guidance of Dr. Floyd Toole and Sean Olive have developed a purely scientific methodology that actually allows them to objectively measure and predict listener preference. This article explores the measurements while also providing the secret formula of the characteristics necessary for producing loudspeakers that score highly in subjective listening tests. This is a must read with a call to action to your favorite loudspeaker company to give us more useful information about their products.

Read: How To Objectively Measure Loudspeaker Performance

Doesn't Soundstage employ the measurement techniques developed by Toole and Olive? That's where Toole did a lot of his research is at the NRC. Also, are you stating that no publication that you know of correlates listener preference to a particular set of measurements? Just trying to get clarification.

 

[gene]

Doesn't Soundstage employ the measurement techniques developed by Toole and Olive? That's where Toole did a lot of his research is at the NRC. Also, are you stating that no publication that you know of correlates listener preference to a particular set of measurements? Just trying to get clarification.

Soundstage only does a portion of what Floyd talks about in this article. Also NOBODY is properly measuring loudspeaker distortion and correlating it to what we hear. CEA 2010 testing is the closest thing but it's being used primarily for subwoofers. We have quite a way to go but the Harman proposal of measurements does level the playing field a good deal and gives enough data on a loudspeaker should one EVER wish to apply EQ above bass frequencies rather than guessing like we are doing now with Auto-EQ and no knowledge of what the speaker is producing vs how the room is influencing it.

 

[3db]

Soundstage only does a portion of what Floyd talks about in this article. Also NOBODY is properly measuring loudspeaker distortion and correlating it to what we hear. CEA 2010 testing is the closest thing but it's being used primarily for subwoofers. We have quite a way to go but the Harman proposal of measurements does level the playing field a good deal and gives enough data on a loudspeaker should one EVER wish to apply EQ above bass frequencies rather than guessing like we are doing now with Auto-EQ and no knowledge of what the speaker is producing vs how the room is influencing it.

Interesting. Does non linear distortion come about from overdriving the drivers' parameters...like say execursion, temperature limits, maybe even drivers playing too loud for their enclosures. I'm trying do understand the differences between linear and non linear distortion as it pertains to drivers and speaker systems as a whole.

I would also have to agree with you that nobody as correlated speaker measurements with what sounds good or bad with maybe the exception of frequency response curves where speakers are described as bright, warm, muddy etc. I have a sneaking suspicion that there's way more to it then that. I would hazard a guess that speaker systems are the most complicated design work in an audio system coupling everything from electrical, mechanical, acoustic, and propagation aspects into a tidy package.

 

[herbu]

a purely scientific methodology that actually allows them to... predict... preference.

Now if they could only come up with one that worked on women...

 

[shadyJ]

Interesting. Does non linear distortion come about from overdriving the drivers' parameters...like say execursion, temperature limits, maybe even drivers playing too loud for their enclosures. I'm trying do understand the differences between linear and non linear distortion as it pertains to drivers and speaker systems as a whole.

Linear distortion is primarily that which affects the frequency response and amplitude. You can think of it as an equalizer. It doesn't depend on the incoming signal. Non-linear response is signal dependent, so you might get distortion in bass or you might get distortion in treble, and so on. To put it another way, linear distortion may change the amplitude of a signal, but not the shape of a signal. Non-linear distortion will change the shape, for example if the signal is a pure sine-wave, you can clip that sine wave and make it resemble a square wave.

Yes, non-linear distortion can come about from over-driving a driver. For example, imagine playing back a sine wave again. When you have pushed that driver to its excursion limits on the sine wave, the spider stops flexing, and that nice smooth round crest of the wave starts to flatten out, in the manner of soft clipping. Of course, when the voice coil former starts smacking into the back plate, that is more like hard clipping.

As for drivers playing to loud for their enclosures, yes, that causes distortion as well. Air within the enclosure has a "spring effect", and sealed enclosures partly rely on this back spring to prevent over-driving. Imagine the back of the driver bouncing off of a higher pressure area, or imagine firmness of a balloon with lots of air in it vs the flacidity of balloon with only a medium amount of air in it. If the enclosure is very small, the air pressure behind the driver will be very high, and not allow as much backward excursion as forward excursion. This uneven excursion will cause harmonic distortion.

As for thermal effects, they cause linear distortion, not non-linear distortion. What happens is that when the voice coil grows warm, it loses its magnetic force because its electrical resistance can grow substantially. This will lower the sensitivity of the driver and also cause sags in bands the frequency response. The permanent magnet can also lose magnetic force by warming up too. Generally when things warm up, they lose their magnetism. The voice coil can heats up much faster than the permanent magnet but it cools relatively swiftly compared to the permanent magnet as well, so when the permanent magnet gets hot, it can affect the sensitivity and output power of the driver for quite awhile.

I would also have to agree with you that nobody as correlated speaker measurements with what sounds good or bad with maybe the exception of frequency response curves where speakers are described as bright, warm, muddy etc. I have a sneaking suspicion that there's way more to it then that. I would hazard a guess that speaker systems are the most complicated design work in an audio system coupling everything from electrical, mechanical, acoustic, and propagation aspects into a tidy package.

There have been lots of ways people have correlated speaker measurements with what sounds good or bad. Floyd Toole's research on room effects and dispersion are some of the most well-known, but there is a ton of research in this area, and a bunch of it specifically relating to distortion types.

 

[gene]

Linear distortion is primarily that which affects the frequency response and amplitude. You can think of it as an equalizer. It doesn't depend on the incoming signal. Non-linear response is signal dependent, so you might get distortion in bass or you might get distortion in treble, and so on. To put it another way, linear distortion may change the amplitude of a signal, but not the shape of a signal. Non-linear distortion will change the shape, for example if the signal is a pure sine-wave, you can clip that sine wave and make it resemble a square wave.

Yes, non-linear distortion can come about from over-driving a driver. For example, imagine playing back a sine wave again. When you have pushed that driver to its excursion limits on the sine wave, the spider stops flexing, and that nice smooth round crest of the wave starts to flatten out, in the manner of soft clipping. Of course, when the voice coil former starts smacking into the back plate, that is more like hard clipping.

As for drivers playing to loud for their enclosures, yes, that causes distortion as well. Air within the enclosure has a "spring effect", and sealed enclosures partly rely on this back spring to prevent over-driving. Imagine the back of the driver bouncing off of a higher pressure area, or imagine firmness of a balloon with lots of air in it vs the flacidity of balloon with only a medium amount of air in it. If the enclosure is very small, the air pressure behind the driver will be very high, and not allow as much backward excursion as forward excursion. This uneven excursion will cause distortion, mainly odd order distortion, so 3rd, 5th, 7th, etc

As for thermal effects, they cause linear distortion, not non-linear distortion. What happens is that when the voice coil grows warm, it loses its magnetic force because its electrical resistance can grow substantially. This will lower the sensitivity of the driver and also cause sags in bands the frequency response. The permanent magnet can also lose magnetic force by warming up too. Generally when things warm up, they lose their magnetism. The voice coil can heats up much faster than the permanent magnet but it cools relatively swiftly compared to the permanent magnet as well, so when the permanent magnet gets hot, it can affect the sensitivity and output power of the driver for quite awhile.



There have been lots of ways people have correlated speaker measurements with what sounds good or bad. Floyd Toole's research on room effects and dispersion are some of the most well-known, but there is a ton of research in this area, and a bunch of it specifically relating to distortion types.

Great post please share the other research on audible distortion you've found. We are actually working on an article regarding perceived distortion in speakers/subs so I'd like to see this info.

The problem with distortion tests I'm seeing now is the simple distortion sweeps done at the NRC are quite primitive. I can run similar tests here on small speakers and as long as they aren't overdriven where the woofer is bottoming out, they pass cleanly with flying colors. You don't see cone break up when for example running a midrange into it's breakup mode b/c the designer didn't put a LPF on the driver. Yet you can clearly hear it happening even at low listening levels.

 

[shadyJ]

The problem with distortion tests I'm seeing now is the simple distortion sweeps done at the NRC are quite primitive. I can run similar tests here on small speakers and as long as they aren't overdriven where the woofer is bottoming out, they pass cleanly with flying colors. You don't see cone break up when for example running a midrange into it's breakup mode b/c the designer didn't put a LPF on the driver. Yet you can clearly hear it happening even at low listening levels.

From what I have been looking at, I agree, the NRC distortion tests are not terribly useful. They only kind of give you an idea of where the speaker is getting stressed. Those distortion tests do not tell you what the speaker sounds like. A flat THD+N measurement isn't worth a whole lot, unless it is extreme, and by extreme I mean like over 80% or less than 1%. Vanilla THD does not even tell you what the speaker sounds like when running a single tone because it does not display the component harmonics, so you can forget about stuff like intermodulation distortion, or anything resembling stuff a human being would voluntarily listen to, like, say music or speech. There have been attempts to popularize distortion metrics which have been demonstrated to correlate much better with human perception (some going all the way back to 1950), but obviously they haven't gained any popular traction.

 

[PENG]

I have to vote yes because of the word "proper" but only in general sense that people would prefer true fidelity sound. In practice, the problem is, what is really "proper"? When that word has a consistent and all agreeable (by experts in the related fields) meaning/definition, that we are in business but even then there are always going to be exceptions because not everyone prefer true fidelity sound. Some prefer colored sound, such as exaggerated bass, treble, mid or any combination of exaggerated or suppressed frequencies bands, or certain kind of harmonic distortions.

 

[Floyd Toole]

I just dropped into this discussion, and as my name has been mentioned I can add some clarification.
First: linear distortion refers to amplitude and phase vs. frequency responses, and it is generally constant over a large dynamic range. However, power compression can change things when the voice coils heat up, but that is considered a "linear" effect too in that it happens slowly, not fast enough to modify waveforms. Linear distortion does not modify the waveform of pure tones - single frequencies - but most certainly modifies the waveforms of complex sounds that contain many frequencies - i.e. everything we listen to. But those modifications are essentially constant over a wide range of signal levels.

Non-linear distortion modifies all waveforms by an amount that depends on the non-linear mechanism and how hard it is driven. It is dependent on signal level. The basic problem is that for different amounts of input signal one gets an inappropriate amount of output signal - the input/output relationship is non-linear. We have no way (yet) to quantify the pure non-linearity in terms of what we hear. Instead, we "probe" the non-linear system with simple signals and see what comes out. With pure tone inputs one gets harmonic overtones that should not be there. Comparing the unwanted harmonics to the wanted (fundamental) signal gives us a percentage harmonic distortion. Almost everyone in the world does this kind of measurement (including the NRC). It tells us something about the system but the numbers do not correlate at all with what we hear in broadband music. Total harmonic distortion that is zero is what we are after. Any amount above zero is undesirable, but we may or may not be able to hear it in music - unless it is very high. Engineers designing transducers find it useful in that it reveals something about the non-linear mechanism that is misbehaving. Because of perceptual masking the signal that causes the distortion prevents us from perfectly hearing lower harmonics, so an improved measure boosts the level for increasingly higher harmonic numbers.

The next step is to use a more complicated signal, twin-tone or multi-tone combinations that generate both harmonic and intermodulation distortion components. Because many of the intermodulation products are well separated from the driving frequencies, especially those lower than the driving frequencies, they are less masked and more easily heard. Hence the common belief that intermodulation distortion is worse than harmonic distortion. Really, they both come from the original problem - the non-linearity - but are simply different ways of trying to quantify it. In the end, again because of perceptual masking (more input signals more masking) this too fails to correlate well with what we hear in music. When the frequency range is subdivided as in two-, three- and four-way systems the scheme falls apart because different driving signals go to different transducers and the numbers cannot be compared between different loudspeakers. However, it is a very good way to demonstrate the advantage of multi-way loudspeakers.

At the present time the best possibility for a useful measurement of non-linear distortion is one that includes models of the key perceptual mechanisms. Because we cannot hear all of the distortion products that are measured, the measurements we have cannot be correct. We have to try to predict what is and is not audible, and that is a massive research project. In the meantime we can all be grateful for perceptual masking - it allows us to think that a lot of non-linear products sound just fine. LPs generate masses of all kinds of distortion and we enjoyed them for years.

 

[highfigh]

Great post please share the other research on audible distortion you've found. We are actually working on an article regarding perceived distortion in speakers/subs so I'd like to see this info.

The problem with distortion tests I'm seeing now is the simple distortion sweeps done at the NRC are quite primitive. I can run similar tests here on small speakers and as long as they aren't overdriven where the woofer is bottoming out, they pass cleanly with flying colors. You don't see cone break up when for example running a midrange into it's breakup mode b/c the designer didn't put a LPF on the driver. Yet you can clearly hear it happening even at low listening levels.

Do you ever send a pure wave to the speakers and use a mic & oscilloscope to monitor the waveform?

 

[Floyd Toole]

OK Gene, the NRC distortion tests are as you say "primitive". Total harmonic distortion (THD) is where distortion measurements begin. There has been no research done at the NRC since a few months after I left about 25 years ago. That the test facility still exists is somewhat miraculous, and it is tended by a part time technician, I'm told. No apologies are necessary.

As for monitoring waveforms on an oscilloscope, I did that first about 50 years ago when I began to look at loudspeaker performance. When I measured harmonic distortion that could be heard in pure tones, the effect on the waveform shape was often simply not visible. But, back then a few pure class B power amps existed. With a pure tone the crossover distortion was easily audible, THD measured about 0.001%, normally thought of as negligible. The tiny blip in the waveform at the zero-voltage transition between the push and pull sections could be seen in the waveform at great magnification, but when the energy was averaged over a whole period of the waveform, it was vanishingly small. So, it became evident that distortion mechanisms that were locked to the waveform might be very differently perceived than those that were long-term averaged over the test signal duration. So, any measurement relying on RMS, energy-averaged, values are telling only part of the story. Was the crossover distortion audible in music? Not that anyone could tell, even with oboe solos in a reverberant concert hall. But that does not excuse it.

ADDITION One of the most difficult forms of distortion to measure, but one of the easiest to hear is what is called "rub and buzz" in which the voice coil is rubbing or some grit is in the gap around it. It generates tiny spikes of energy, correlated with cone movements - i.e. waveforms - but hardly shows up in energy-averaged distortion measurements.

As I said in my last post, the distortion one measures is totally related to the test signal used to probe the non-linearity. With music and movies that is infinitely variable.

The THD measurement was left in place because it happened almost automatically and a few loudspeakers would exhibit narrow-band distortion spikes when they misbehaved. It was left in the test repertoire for that reason only, not because this simple metic has any great meaning. We need a metric that can eliminate or attenuate distortion components that are perceptually masked. Then the numbers have a chance of telling us something useful. In the meantime, the closer to zero distortion, the better.

 

[gene]

OK Gene, the NRC distortion tests are as you say "primitive". Total harmonic distortion (THD) is where distortion measurements begin. There has been no research done at the NRC since a few months after I left about 25 years ago. That the test facility still exists is somewhat miraculous, and it is tended by a part time technician, I'm told. No apologies are necessary.

As for monitoring waveforms on an oscilloscope, I did that first about 50 years ago when I began to look at loudspeaker performance. When I measured harmonic distortion that could be heard in pure tones, the effect on the waveform shape was often simply not visible. But, back then a few pure class B power amps existed. With a pure tone the crossover distortion was easily audible, THD measured about 0.001%, normally thought of as negligible. The tiny blip in the waveform at the zero-voltage transition between the push and pull sections could be seen in the waveform at great magnification, but when the energy was averaged over a whole period of the waveform, it was vanishingly small. So, it became evident that distortion mechanisms that were locked to the waveform might be very differently perceived than those that were long-term averaged over the test signal duration. So, any measurement relying on RMS, energy-averaged, values are telling only part of the story. Was the crossover distortion audible in music? Not that anyone could tell, even with oboe solos in a reverberant concert hall. But that does not excuse it.

As I said in my last post, the distortion one measures is totally related to the test signal used to probe the non-linearity. With music and movies that is infinitely variable.

The THD measurement was left in place because it happened almost automatically and a few loudspeakers would exhibit narrow-band distortion spikes when they misbehaved. It was left in the test repertoire for that reason only, not because this simple metic has any great meaning. We need a metric that can eliminate or attenuate distortion components that are perceptually masked. Then the numbers have a chance of telling us something useful. In the meantime, the closer to zero distortion, the better.

Thanks Floyd. Oscilloscopes are not good devices for seeing distortion until the distortion levels are too high (you only get about 40db resolution on an analog scope or 1%).

I would think a multi-tone test as you suggested would be the way to go. Perhaps something like Don Keele developed for subwoofers in the CEA 2010 protocol. If you have a suggestion regarding a waveform that should be used to be a close approximation to music program material in the frequency bands our ears are most sensitive to, I am sure it can be created to run on my Audio Precision. I know the folks at AP are really trying to get into the loudspeaker measurement business with their software on their APX585 Audio Analyzers but they have a long way to go. Of course we'd have to have mass adoption of such a signal and testing criteria and you know what a challenge that is in this industry

Have you seen how Linkwitz Labs measures distortion for loudspeaker drivers?

A very effective test signal for measuring both harmonic and intermodulation distortion is a sinewave that is 100% amplitude modulated by another sinewave at 1/10th its frequency.​

See: http://www.linkwitzlab.com/mid_dist.htm

 

[Floyd Toole]

Gene, as I said earlier: "the distortion one measures is totally related to the test signal used to probe the non-linearity. With music and movies that is infinitely variable."

Siegfried was comparing similar drivers to find the best one, and chose a signal that generated a lot of distortion products - the winner would exhibit the lowest distortion. There is nothing wrong with that. As I also said earlier: Zero distortion is the only certain goal, but above zero distortion is the reality.

The problem is that we want to relate measured distortion numbers - above zero - to how audible or how annoying the distortion is. That is the tricky part because of perceptual masking: all of the distortion products that are generated and measured are not audible. Therefore the numbers are not reliable indicators of what we hear.

The problem is the input/output moment-by-moment non-linearity. Some of the effects of this non-linearity can be quantified by probing/stimulating the non-linear system with known signals and measuring how much they have changed at the output. Until we have a way to identify those added distortion products that have a potential of being audible from those that are not likely to be heard, the numbers will be wrong. There can be no "magic bullet" test signal. We need a "magic bullet" distortion analyzer.

I discuss this in Section 19.3 in my book. It is a short section because there is really little to say - we simply do not have the answer. More words would not help. Dr. Alex Voishvillo (a Harman employee, BTW) wrote what I consider to be one of the best analyses of distortion measurement (AES preprint 6910), concluding that perceptual models in the distortion analyzers are the key to useful numbers. Other serious investigators agree - they are referenced in the book.

The good news is that, in the real world, it is a rare event for non-linear distortion to be a factor in our listening pleasure. It happens though , and simple distortion measures (e.g. THD) are probably adequate to identify the gross offenders which is a reason to continue doing them. It is the subtle offenders that we cannot reliably identify. So, loudspeaker designers seek out transducers with the lowest distortion, and cross their fingers. Mostly it works very well.

 

[TheWarrior]

I discuss this in Section 19.3 in my book. It is a short section because there is really little to say - we simply do not have the answer. More words would not help. Dr. Alex Voishvillo (a Harman employee, BTW) wrote what I consider to be one of the best analyses of distortion measurement (AES preprint 6910), concluding that perceptual models in the distortion analyzers are the key to useful numbers. Other serious investigators agree - they are referenced in the book.

Floyd, this thread and others, have demonstrated how clearly you can convey complex information. I ordered your book because of this. Since I don't have a degree in engineering, I doubt I'll ever use the knowledge professionally. But I do know my wife and I will reap the benefits! Thank you!

 

[Floyd Toole]

Thank you "Warrior". I try to demystify a topic that for far too long has been shrouded in mystery and mystique. There is a real scientific foundation underlying audio, but it is not well known, and there are people who see a need to keep it that way. They tend to earn their livings by promoting their opinions or their products, so in one sense you can't blame them, but in the end they are only opinions . . .

And no engineering degree is necessary to read and understand my book. I took great pride in keeping the math out of it. Frankly, it is not necessary to do what is necessary to achieve great musical satisfaction. Mostly, just as in live performances, if you have a good instrument and a good musician, satisfaction will follow. Here, if you are fortunate enough to have good loudspeakers to reproduce good recorded sound, you are most of the way to satisfaction. Little else is necessary.

Check out my recent YouTube lecture for a painless introduction to the science of audio. It shows that many of the problems begin in recording studios.

Enjoy!

 

[highfigh]

Check out my recent YouTube lecture for a painless introduction to the science of audio. It shows that many of the problems begin in recording studios.

Enjoy!

Your comments about the differences heard in various recordings being the result of using different monitors is something I have explained to people a lot since I started selling audio equipment in the late-'70s. I have also heard a lot of people say that they want their system to sound exactly the way it did when the music was mixed and I reminded them that they would have to assemble the same system, in the same room, with the same effects, etc.

I have to think that a lot of the bad-sounding recent music releases are due to independent label releases, using one budget-priced speaker as the monitor, in a place where the response isn't known and faults aren't corrected before they mix and EQ.

 

[Floyd Toole]

Yes it is quite likely that poor loudspeakers in home studios are responsible for a lot of flawed sound. But bad loudspeakers have always been around, even now in some supposedly prestigious studios. On top of this, there is the guessing game of "room EQ", which is rampant in both consumer and pro rooms.

This problem also exists in the film sound business. We can all be grateful that humans are so adaptable - and forgiving - to allow the music itself to be enjoyed.

 

[mtrycrafts]

Your comments about the differences heard in various recordings being the result of using different monitors is something I have explained to people a lot since I started selling audio equipment in the late-'70s. I have also heard a lot of people say that they want their system to sound exactly the way it did when the music was mixed and I reminded them that they would have to assemble the same system, in the same room, with the same effects, etc.

I have to think that a lot of the bad-sounding recent music releases are due to independent label releases, using one budget-priced speaker as the monitor, in a place where the response isn't known and faults aren't corrected before they mix and EQ.

That would require a lot of speakers and listening rooms to recreate all those different recording from different companies.

 

[TheWarrior]

And no engineering degree is necessary to read and understand my book. I took great pride in keeping the math out of it. Frankly, it is not necessary to do what is necessary to achieve great musical satisfaction. Mostly, just as in live performances, if you have a good instrument and a good musician, satisfaction will follow. Here, if you are fortunate enough to have good loudspeakers to reproduce good recorded sound, you are most of the way to satisfaction. Little else is necessary.

Check out my recent YouTube lecture for a painless introduction to the science of audio. It shows that many of the problems begin in recording studios.

Enjoy!

Is there information on dealing with awkward room boundaries? I was kind of hoping there were a few equations I could plug variables in to. My eyes don't glaze over when I see the Calculus. I just don't want to have to memorize the equations for a test, but I know enough to make use of those formulas!