Audio Measurements: The Useful vs. the Bogus in Consumer Audio

[gene]

RE: Waterfall Plots by Dr. Floyd Toole

I had a discussion about the usefulness or lack thereof of Waterfall Plots with Dr. Floyd Toole. He allowed me to post his email response to me which I copied below:

Waterfall plots are wonderfully ornamental, but rarely instructive. Because loudspeaker drivers are minimum-phase devices, their time-domain performance is predictable from the steady-state amplitude response. The most revealing (highest resolution) amplitude response is the steady-state response obtained in an anechoic environment. Waterfall plots require time-windowed FFT data and as a result what one can see in both the frequency and time domains depends on the duration of the window - at one extreme the window is infinite, and we see a perfect amplitude response, but see nothing in the time domain, at the other extreme the window is short and we see "something" in the time domain, but nothing useful in the frequency domain. My money is on the steady-state response for loudspeaker testing, but I fully understand the attraction of waterfalls for "marketing" purposes.

I explain some of this in and around Figure 13.23 in my book. There I am using LF room measurements, where one can actually learn something because rooms are much more complicated than speakers. However, that said, as shown in my figure, it is very easy to be mislead by waterfalls, and the literature, including lots of reviews and manufacturer's claims, is full of examples in which what is claimed cannot possibly be true. I see a lot of waterfalls that look something like Figure 13.23b, suggesting huge problems, when there in fact may be a minor problem, or none at all. The people making the measurements don't understand the limitations of the measurement. Making it all worse, when such waterfalls are shown, the duration of the measurement time window is not revealed, so we cannot interpret the meaning of the data in either the frequency or time domain. It is show-biz, not engineering.

In short, if the steady-state amplitude response of a loudspeaker shows a high-Q peak, there will be ringing in the waterfall. If it does not, there will be no ringing, and everything in between. So what? There is no new information - but it is ornamental.

 

[mtrycrafts]

...I actually learned a lot writing this article and conversing directly with Dr. Toole. ...

You spoke with the speaker god?
You should invite him to join us here.
After all, he is retired with all the time in the world on his hands

As a result, numerous loudspeaker companies (that actually welcome criticism) have made product changes after our reviews, others have simply went away and/or cancelled advertising contracts

As the saying goes, or something like it 'don't let facts get in the way'

Re: Power Cube Measurements

Maybe David Rich, the inventor of it, will sell you his load box with the different loads and phase shifters at a reasonable $$$ Then, it would be quickly done. His picture surely gives you a quick idea of sags at those testing areas.

 

[mtrycrafts]

That's not what I'm saying. What I'm saying is that the interaction between speakers and the room acoustics plays a very big part on a how a speaker sounds. So good off axes response on a speaker in an anoechic chamber may play real havoc in a room with highly reflective walls etc and sound very poor as a result.

Just think of how another speaker that is not so good in the chamber will respond in that same room. Terribly. I doubt the room on its own can improve a speakers performance.

With a standard measuring methodology as in the chamber, at least you know what that speaker is capable of. The room interaction is a separate issue that the owner needs to measure and deal with.

In room measurements would only useful in that room, in that spot and have no real meaning to anyone else.

 

[mtrycrafts]

Over the years, it has become quite clear that measuring devices are a tool for the reviewer and manufacturer who tries, in this manner, to differentiate the reviewed equipment from others. From my experience, whether it be audio or video, little of this has to do with "real world" applications. The example of the SMS-1 is somewhat flawed in that for any bass measuring device, the human ear does not hear accurately below 50HZ anyway so this requires manual adjustment to compensate for this so really, what shows as a smooth grid on the screen, (and it very well may be "accurate"), may not though, necessarily represent what the ear hears(or likes). The same goes for video.

As always, let your eyes and ears be the finally arbiter of what looks or sounds good.

Which reviewer uses measuring devices, excluding this web site, in reviewing? Stereophile adds the measure data but I doubt the reviewer himself cares about it as he waxes on about his perceptions, accurate or not.

Without measurements, it is chaos out there.
Interestingly, Toole's research, and others at NRC have shown what people really prefer when bias is removed. They prefer speakers that measure very well.

 

[ratso]

i think i learned more in one day about speakers by reading dr. toole's papers than i learned in a lifetime up to that point. here are a few of them - BTW a great deal of them are for laymen and not too techno-babble-ish.

http://www.infinitysystems.com/home/technology/technology_whitepapers.aspx?Language=ENG&Country=US&Region=USA

 

[mtrycrafts]

i think i learned more in one day about speakers by reading dr. toole's papers than i learned in a lifetime up to that point. here are a few of them - BTW a great deal of them are for laymen and not too techno-babble-ish.

Yes, it does pay to listen to the experts.

 

[MinusTheBear]

Yes, it does pay to listen to the experts.

Yes it does. With the Bose cube budget, you can outfit your living room with Infinity Primus and ONE HELL OF A NIGHT at the Canadian ballet.

 

[lejack]

The article makes much discussion of THD, but what about the more revealing IMD (intermodulation distortion, for the newbies). Years...decades?...ago, it was commonly measured.

 

[3db]

Just think of how another speaker that is not so good in the chamber will respond in that same room. Terribly. I doubt the room on its own can improve a speakers performance..

A speaker with limited dispersion will sound better in a live room compared to that of a speaker with a wide dispersion.

With a standard measuring methodology as in the chamber, at least you know what that speaker is capable of. The room interaction is a separate issue that the owner needs to measure and deal with.

In room measurements would only useful in that room, in that spot and have no real meaning to anyone else.

I have not nor ever have adovcated in room measurements. What I'm been stating all this time is that speakers that may measure really well could end up sounding really bad in a poor acoustic environment...hence I think in home auditions hold much more creedance than just a spec sheet.

 

[GranteedEV]

A speaker with limited dispersion will sound better in a live room compared to that of a speaker with a wide dispersion.

I'd say the most important thing is the uniformity of that dispersion. Of course, with an extremely wide dispersion speaker, placement is probably more sensitive.

 

[AcuDefTechGuy]

speakers that may measure really well could end up sounding really bad in a poor acoustic environment

By "poor acoustic environment", do you mean a room made of glass or metal or just a standard average living room with sofas, carpet, and curtains?

I think a speaker that measures extremely well both on-axis and off-axis will sound extremely good in the standard average living room with sofas, carpet, and curtains.

 

[gene]

woohoo 180dB measurement scales

http://www.hometheaterhifi.com/subwoofers-products-menu-column1-41/1260-sunfire-sds-10-subwoofer.html?start=3

 

[jliedeka]

They don't measure everything but when they do, they seem to know what they are doing.

In my latest issue of Home Theater mag, I noticed in the speaker review they used a 40dB scale and didn't say if they were smoothing although I would guess it was 1/3 octave. They also just present one graph that is an average. Doesn't tell you much. I think, like Stereophile, they use measurements to justify their subjective reviews.

Jim

 

[gene]

They don't measure everything but when they do, they seem to know what they are doing.

Umm 180dB scale is NOT a good thing.

 

[3db]

Umm 180dB scale is NOT a good thing.

WHAT?? I CAN"T HEAR YOU!!! TURN IT DOWN

 

[jliedeka]

I just realized that was a distortion graph, not a response graph which I thought looked horrible. I don't like that style of graph, I prefer one that shows the various harmonic related to the fundamental like the Soundeasy ones Zaph posts.

Jim

 

[haraldo]

Thanks Gene for great article....

As always, what you write is second to none quality and contents

 

[haraldo]

I never trusted a company like Axiom that recommends using 2 center speakers anyway.

But that's another story.

Sorry.

Sorry to go off-topic.... but some trolls have two heads

 

[highfigh]

Umm 180dB scale is NOT a good thing.

It is if you want to recreate a Space Shuttle liftoff.

 

[gene]