Audio Measurements: The Useful vs. the Bogus in Consumer Audio

[admin]

Let's face it we all like things to run smooth. It's every audiophiles dream to own a loudspeaker that measures ruler flat from 20Hz to 20kHz. Many audio magazines love to regurgitate the manufacturers claims either in text or their own measurements to emphasize product positives. It's my belief that most of the manufacturers and audio magazines aren't flat out lying or being deceptive as much as they aren't giving you the whole story likely because of incompetence or lack of proper measurement facilities at their disposal. It's easier to furnish pretty graphs because quite frankly to paraphrase Jack Nicholson "you (consumers) can't handle the truth". In most cases these measurements do have a grain of truth, if you place a lot of conditions and caveats on them. This is often not disclosed, but it's my goal to do so in this article where we explore various measurement and graphing techniques. Can you handle the truth? If so, read on...

Discuss "Audio Measurements: The Useful vs. the Bogus in Consumer Audio" here. Read the article.

 

[jliedeka]

Good article. A lot of us are probably aware of at least some of the issues you brought up but it was great to see a summary of common measurement problems.

I also highly recommend Floyd Toole's book. It's required reading for anyone who has an interest in achieving good sound reproduction.

Jim

 

[MinusTheBear]

Here is an example of a manufacturers graph of a speaker and the problems that are associated with it. Textbook example for this article.

http://www.axiomaudio.com/gallery_disp.html?image=M80_freq.gif&title=M80

1) 120 dB scale
2) It looks like there is some kind of smoothing applied that is not disclosed.
3) It doesn't state what type of measurement only SPL vs Freq (on-axis, listening window).
4) Doesn't state the criteria and how the measurement was conducted.

 

[MinusTheBear]

Now here is the opposite of another manufacturer that shows meaningful measurements that will give you a good idea how their speaker might sound and how difficult the load it might present( impedance shown with electrical phase) with the amplifier. On-Axis, Listening window, Vertical/Horizontal Off-Axis are all good indicators. However, this manufacturer even goes as far to show the CSD (waterfall plot).

http://www.ascendacoustics.com/pages/products/speakers/SRM1/srm1meas.html

Note:

1) 50 dB scale
2) Unsmoothed measurements.
3) States the conditions and how each measurement was performed.
4) States what each measurement is.

 

[GranteedEV]

I was just about to mention the same thing. Is there a particular reason Audioholics doesn't post waterfalls? I would guess because it's difficult in a room with reflections.

My list of relevant measurements would be

-180 degree Off-axis response in small intervals / Polar Response / Directivity Index / summed power response whatever you want to call it or how you want to describe it. The speaker's interaction with the room and listening position(s). This can especially reveal crossover /sweet spot issues IMO. I think audioholics could improve on these measurements to be honest.

-Cumulative Spectral Decay. Checking for resonances and ringing that can lead to harshness, as well as perception over time. ON that note, how relevant is something like step response? Either way, it's pretty easy to recognize disaster speakers with waterfall graphs.

-Thermal Compression + harmonic distortion + IMD of the speaker itself at high SPL rather than of the amp. On that note, isn't there a way to measure inductor linearity? Considering half the speakers reviewed use iron cores at bass frequencies it seems logical to know about their behaviour.

-Impedance/ Electrical Phase / sensitivity for obvious reasons, especially since most people don't want to have to buy Mark Levinson monoblocks to drive their $350 speakers

-Flat On-axis Frequency response. I think it's kind of overated since most half decent speakers already strive for this in their design, but it can definitely reveal some things about really bad speakers, just like a CSD.

I guess we'd also be interested in acoustic phase response, although I don't know how relevant such a thing is.

Regarding amps, I also wonder if maybe audioholics should be doing powercube measurements, which test the amp into a resistive load but rather into varying loads of impedance and phase angle. This is what the audio critic has done before and it seems to be a very relevant measurement IMO, although likely not all too flattering.

I'd also like to see some indication of "minimal" crossover distortion in Class AB amp testing. I think you can see this by simply analyzing the sine waveform but I'm not sure.

 

[3db]

I see alot of manufacturers offer off axes frequency response. I'm assuming these off axes measurements are taken in an aneohic chamber. However, the real life rooms that these speakers are placed in will have a dramatic effect on how these speakers will sound. Even the on axes response will be affected by the floor that the speakers are placed on. So does specing the hell out of the speakers in anoechic chamber really going to paint a good picture in a real life room for the average consumer? I'm not saying abandon specifications of speakers properly. What I am saying is (because Ive seen posts before about people buying speakers on specs alone ) purchasing a loudspeaker on specs alone is analgous to playing Russian roulette with 5 chambers loaded. Just becuase a manufacture specs better than another is not indicative of how teh speaker is going to sound in a real life room.

 

[MinusTheBear]

I see alot of manufacturers offer off axes frequency response. I'm assuming these off axes measurements are taken in an aneohic chamber. However, the real life rooms that these speakers are placed in will have a dramatic effect on how these speakers will sound. Even the on axes response will be affected by the floor that the speakers are placed on. So does specing the hell out of the speakers in anoechic chamber really going to paint a good picture in a real life room for the average consumer? I'm not saying abandon specifications of speakers properly. What I am saying is (because Ive seen posts before about people buying speakers on specs alone ) purchasing a loudspeaker on specs alone is analgous to playing Russian roulette with 5 chambers loaded. Just becuase a manufacture specs better than another is not indicative of how teh speaker is going to sound in a real life room.

According to researchers like Sean Olive, Floyd Toole and many expert speakers designers think so when combined with controlled listening tests. The really important measurement we never see however is the Total Radiated Sound Power Response. This is used to represent how sound arrives after encountering more than one boundary.

 

[MinusTheBear]

Perhaps Gene can comment on this about the influence of the room.

One trend I noticed in AH's loudspeaker reviews is that the really good speakers are always noted to play nice in room and have no serious audible flaws in room. I have come to the opinion recently that if a speaker is really picky to placement and/or are experiencing things like fat bass, brightness or listener fatigue in a commonly furnished household living room, the problem probably isn't the room but that of a flawed speaker.

 

[gene]

I was just about to mention the same thing. Is there a particular reason Audioholics doesn't post waterfalls? I would guess because it's difficult in a room with reflections.

My list of relevant measurements would be

-180 degree Off-axis response in small intervals / Polar Response / Directivity Index / summed power response whatever you want to call it or how you want to describe it. The speaker's interaction with the room and listening position(s). This can especially reveal crossover /sweet spot issues IMO. I think audioholics could improve on these measurements to be honest.

-Cumulative Spectral Decay. Checking for resonances and ringing that can lead to harshness, as well as perception over time. ON that note, how relevant is something like step response? Either way, it's pretty easy to recognize disaster speakers with waterfall graphs.

-Thermal Compression + harmonic distortion + IMD of the speaker itself at high SPL rather than of the amp. On that note, isn't there a way to measure inductor linearity? Considering half the speakers reviewed use iron cores at bass frequencies it seems logical to know about their behaviour.

-Impedance/ Electrical Phase / sensitivity for obvious reasons, especially since most people don't want to have to buy Mark Levinson monoblocks to drive their $350 speakers

-Flat On-axis Frequency response. I think it's kind of overated since most half decent speakers already strive for this in their design, but it can definitely reveal some things about really bad speakers, just like a CSD.

I guess we'd also be interested in acoustic phase response, although I don't know how relevant such a thing is.

Regarding amps, I also wonder if maybe audioholics should be doing powercube measurements, which test the amp into a resistive load but rather into varying loads of impedance and phase angle. This is what the audio critic has done before and it seems to be a very relevant measurement IMO, although likely not all too flattering.

I'd also like to see some indication of "minimal" crossover distortion in Class AB amp testing. I think you can see this by simply analyzing the sine waveform but I'm not sure.

Re: Waterfall plots
I'm not a huge fan of waterfall plots. They look cool but they don't really give you anything more useful than properly done frequency response measurements. If the speaker behaves in the frequency domain it should also do so in the time domain. An impulse response can also be taken to look for any resonance issues. Personally I run a very slow sweep on speakers and listen for cabinet resonances or driver break up. This is how I often find audible breakup modes in many metal dome tweeters.

I actually learned a lot writing this article and conversing directly with Dr. Toole. I will start doing spatially averaged plots in future speaker reviews as a result.

Suffice it to say no matter how many loudspeaker measurements we make, you almost never get the whole picture. Very few people do dynamic compression tests in loudspeakers. I do them mostly in-room which is accurate above 500Hz but there is a problem when a loudspeaker employs a PTC device to protect the tweeter and you run steadstate signals into the speaker which activates the device. With normal program material the PTC will likely never trip but they almost always do when running continuous sinewave sweeps. I suppose pinknoise burst testing would be a good alternative but there really is no standard for measuring this.

This is why I test all the loudspeakers I review in my 6,000ft^3 theater room. I always find loudspeaker flaws and limitations in my torture tests. As a result, numerous loudspeaker companies (that actually welcome criticism) have made product changes after our reviews, others have simply went away and/or cancelled advertising contracts



Re: Power Cube Measurements
I do such comprehensive testing of amps now that if I added these tests, I'd have to take others away else it would take ungodly amounts of time to test a single product.

While power cube testing is cool, I see little need for doing these tests since I thoroughly test amps under various loading and power testing conditions. Amp misbehavior would surface under these tests. I also blast them in my huge room with real program material driving real 8 and 4 ohm speakers.

It's possible in the future I may expand on our amp testing but its pretty solid as is right now.

 

[3db]

According to researchers like Sean Olive, Floyd Toole and many expert speakers designers think so when combined with controlled listening tests. .

Please clarify your statemment. Controlled in what way? DBT tests?

 

[MinusTheBear]

Please clarify your statemment. Controlled in what way? DBT tests?

The most common type of controlled listening test would be that of a single blind where the listening panel does not know what they are listening to but the manufacturer/experimenter choosing and setting up the test is aware of their own products and competing products under the test along with conducting their own analysis.

 

[3db]

The most common type of controlled listening test would be that of a single blind where the listening panel does not know what they are listening to but the manufacturer/experimenter choosing and setting up the test is aware of their own products and competing products under the test along with conducting their own analysis.

That's not what I'm saying. What I'm saying is that the interaction between speakers and the room acoustics plays a very big part on a how a speaker sounds. So good off axes response on a speaker in an anoechic chamber may play real havoc in a room with highly reflective walls etc and sound very poor as a result.

 

[MinusTheBear]

That's not what I'm saying. What I'm saying is that the interaction between speakers and the room acoustics plays a very big part on a how a speaker sounds. So good off axes response on a speaker in an anoechic chamber may play real havoc in a room with highly reflective walls etc and sound very poor as a result.

Controlled listening tests do not take place in anechoic chambers but in a actual room. This is how you can achieve a correlation between the two.

 

[AcuDefTechGuy]

I thought Dr. Toole said somewhere that a speaker with great off-axis response will be significantly less affected by the room acoustics?

Linkwitz said the same thing?

 

[3db]

Controlled listening tests do not take place in anechoic chambers but in a actual room. This is how you can achieve a correlation between the two.

What does this have to do with room acoustics? Please site my last example about good off axes response in a room with highly reflective surfaces in your explanantion.

 

[MinusTheBear]

What does this have to do with room acoustics? Please site my last example about good off axes response in a room with highly reflective surfaces in your explanantion.

From what I gather, speakers that measure well anechoically among a family of curves not just off-axis response don't have as many drastic room related problems.

Somebody like Sean Olive could explain this better than me if I am correct or wrong.

 

[GranteedEV]

I thought Dr. Toole said somewhere that a speaker with great off-axis response will be significantly less affected by the room acoustics?

If you look at the speakers that I guess we associate with Dr. Toole`s research, it`s something like the LSR6332:

http://www.jblpro.com/catalog/support/getfile.aspx?doctype=3&docid=569

Those, I guess, would be the measurements JBL deems important. Definitely some marketing department in play on those measurements though

That's not what I'm saying. What I'm saying is that the interaction between speakers and the room acoustics plays a very big part on a how a speaker sounds. So good off axes response on a speaker in an anoechic chamber may play real havoc in a room with highly reflective walls etc and sound very poor as a result.

I think that if the crossover region does not have matching directivity, you're going to hear the issues in the off axis response. Here is an example of a popular beloved speaker that has bad polar measurements:

Not flattering at all!

But when the polar off-axis is essentially the same as the polar on-axis from to 8khz and higher:

What you get is a speaker that revels in the room behavior (pun... that speaker is a Revel )

You want reflections actually, if they sound the same they add spaciousness. THey're not going to color the sound if they're the same (just later and lesser in amplitude).

The only reflections we don't really want are the first and maybe second reflection in most rooms (first maybe 8-10 ms), because they take away from the sharpness in the imaging.

That's what Geddes' speakers are all about. Extremely even polar response, but narrow directivity designed for a very reflective room. that way you don't need absorption panels hurting the sound.

Here is the polar of the Geddes Summa:

Just like toole's research, out to 60 degrees the off axis response is EXCEPTIONALLY similar, at which point it drops 6db in the imaging frequencies towards 90 degrees off axis. This makes it a good idea for most rooms.

It does have a shift to omni response in the bass frequencies. This is the case with all monopolar speakers. It also has a hole in its on-axis response. If you measured just the on-axis you would think it sucks. But that is a result of the Oblate Spherioid waveguide used causing some on axis cancelation. The speaker itself is made for reasonably flat frequency response at 22.5 degrees off axis. Geddes logic is that his speakers are not toed into the listener's ear but rather in front of the listeners - so actually, the listening window response is flat, just not the official on axis response. And the listening windoer/sweet spot is extremely wide and uncolored. Imaging is sharp because there's no early reflections.

An alternative is a fully omni speaker, WMAX style. The placement etc for an omni is probably really strict, but since the radiation in all directions will be so even, it will probably sound really good.

Returning to the revel, it probably has wider dispersion than the geddes speaker. So you will probably "hear the room" more. But again, that's not necessarily a bad thing.

It's all a tradeoff. The number one thing is to start with a speaker with smooth, consistent off axis response - worry about the rest later

 

[kevon27]

Yes, another excellent article from Audioholics I don't understand with the technical talk.. Excellent..

 

[AcuDefTechGuy]

Here is an example of a speaker that has bad polar measurements:

But when the polar off-axis is essentially the same as the polar on-axis from to 8khz and higher:

What you get is a speaker that revels in the room behavior (pun... that speaker is a Revel )

So the 2nd example is a Revel Salon 2 speaker that revels in the room behavior?

And the 1st example is a B&W 802D speaker that sucks in the room behavior?

 

[Casey01]

Over the years, it has become quite clear that measuring devices are a tool for the reviewer and manufacturer who tries, in this manner, to differentiate the reviewed equipment from others. From my experience, whether it be audio or video, little of this has to do with "real world" applications. The example of the SMS-1 is somewhat flawed in that for any bass measuring device, the human ear does not hear accurately below 50HZ anyway so this requires manual adjustment to compensate for this so really, what shows as a smooth grid on the screen, (and it very well may be "accurate"), may not though, necessarily represent what the ear hears(or likes). The same goes for video.

As always, let your eyes and ears be the finally arbiter of what looks or sounds good.