Can Objective Loudspeaker Measurements Predict Subjective Preferences?

[Art Vandelay]

While I did not know how many speaker manuafacturers were specing distortion, I did know B&W does. In any case, my main point in this a case was more about understanding the system. Not much point in pursuing amplifiers that have really low distortion, when your speakers are order of magnitude worse. Anyway, good choice on your part.

FTR I performed my own measurements with an Earthworks microphone and a low distortion analyser.

As for measuring speakers in a room, distortion is more of a relative measurement. Frequency response is clearly more of a direct measure and while the real challlenge is at lower frequencies, you make a interesting point about the impact of ceiling reflections. While they do impact FR measurement, there are major questions over their audibilty. If you have not already done so, suggest you read this https://www.audioholics.com/room-acoustics/room-reflections-human-adaptation or better yet, get Dr Toole’s book.

I haven't read the entire book but I have read excerpts as well as many of his forum posts. Like most people with an audio system I've spent a reasonable amount of time trying to get the best possible sound at the listening chair, and to that end I regard his work as an invaluable reference. It definitely can't hurt to have some understanding of speaker and listening room interactions, to the point of being able to predict what tonal colorations are likely, but we're also fortunate these days to have an abundance of low cost tools to perform measurements and simulations.

In a more simple way, consider a performer playing a violin in your room. If your room treatment really improved the sound quality of the live performer, it may be worthwhile.

But how valid is the comparison between an instrument and a speaker, when the two radiate sound energy in very different ways?

FWIW, I constructed my own convolver filter from live measurements to correct a dominant room mode at 38Hz and it made a big difference; removing a bass boom issue that was negatively impacting many recordings. However, high frequency reflections are not necessarily as measurable as they are audible, due to the fact that they impact more on time domain performance.

I tend not to use much in the way of sound absorption treatment, but there are curtains, carpet and sofas etc that do reduce mid and high frequency reflections.

 

[Winkleswizard]

FTR I performed my own measurements with an Earthworks microphone and a low distortion analyser.




I haven't read the entire book but I have read excerpts as well as many of his forum posts. Like most people with an audio system I've spent a reasonable amount of time trying to get the best possible sound at the listening chair, and to that end I regard his work as an invaluable reference. It definitely can't hurt to have some understanding of speaker and listening room interactions, to the point of being able to predict what tonal colorations are likely, but we're also fortunate these days to have an abundance of low cost tools to perform measurements and simulations.



But how valid is the comparison between an instrument and a speaker, when the two radiate sound energy in very different ways?

FWIW, I constructed my own convolver filter from live measurements to correct a dominant room mode at 38Hz and it made a big difference; removing a bass boom issue that was negatively impacting many recordings. However, high frequency reflections are not necessarily as measurable as they are audible, due to the fact that they impact more on time domain performance.

I tend not to use much in the way of sound absorption treatment, but there are curtains, carpet and sofas etc that do reduce mid and high frequency reflections.[/QUOTE

My original posting was referring to room/speaker frequency response measurements. So I took your comments about reflections in that measurement context. You will have a first reflection from the ceiling (notably for a typical eight foot one). This does affect FR measurement. If you did your ceiling treatment for some other reason, that was not clear to me. Toole’s book does not discuss much about ceiling treatment, but does state “...numbers and graphs are not always simply or logically related to what we hear”.

This plays to my point in the context of the OP. I can tell you I have heard great sound in setups that were never measured and had no room treatment or any frequency/time domain processing. While there is some substantial science about improving sound at low frequencies, beyond that, the answers are more complex as recording quality and psychoacoustics play more significant roles. In this case however, reflections are an essential part of how we localize sound. This is true whether the source is a speaker, a musical instrument or a mechanical noise. So, while in favor of science, when there is scientific uncertainty, I admit I rely on what sounds good to me.

Ww

 

[Art Vandelay]

My original posting was referring to room/speaker frequency response measurements. So I took your comments about reflections in that measurement context. You will have a first reflection from the ceiling (notably for a typical eight foot one). This does affect FR measurement. If you did your ceiling treatment for some other reason, that was not clear to me. Toole’s book does not discuss much about ceiling treatment, but does state “...numbers and graphs are not always simply or logically related to what we hear”.

Ceiling reflections might be in the range 6 -10 dB below the direct path signal, but they don't necessarily make a huge difference to measurements at the listening position when FR is averaged over a region, which is how it's typically done. I think there's been an assumption that ceiling reflections are less problematic because the ceiling is further away from a tweeter than the side walls in a typical compact listening room, but in rooms where that distance is more similar it stands to reason that ceiling reflections will be a contributor to the tonal balance, if for no other reason than the fact that most speakers exhibit a very uneven frequency response at angles >30 degrees above the tweeter axis. It's really just a rearrangement of the argument that speakers which exhibit a more uniform polar response in the horizontal plane will sound more accurate in a typical small room.

This plays to my point in the context of the OP. I can tell you I have heard great sound in setups that were never measured and had no room treatment or any frequency/time domain processing. While there is some substantial science about improving sound at low frequencies, beyond that, the answers are more complex as recording quality and psychoacoustics play more significant roles. In this case however, reflections are an essential part of how we localize sound. This is true whether the source is a speaker, a musical instrument or a mechanical noise. So, while in favor of science, when there is scientific uncertainty, I admit I rely on what sound good to me.

I actually voted for option 3, because in the end our ears and brain make the best fft analyser.

 

[Winkleswizard]

Ceiling reflections might be in the range 6 -10 dB below the direct path signal, but they don't necessarily make a huge difference to measurements at the listening position when FR is averaged over a region, which is how it's typically done....

Yes, just realized a significant difference in what we are measuring. I use REW to measure a single speaker in a room. So am usually gating a measurement based on a ceiling reflection. The goal in this case is to remove the effect of the ceiling interaction from the measurement and it does make a significant difference. I gather you are measuring different speakers from one or more listening positions with the goal of determining how room modes affect the overall sound. This would be a lot more work to do well. To equalize meaningfully at frequencies above 300 Hz, Toole states that (the notion the traditional 1/3 octave analyzer) can reliably predict what is perceived by two ears and a brain is “preposterous”. Since I am pretty sure that at least 2 of my AVRs attempt to do just this, I only use the DSP for home theater and use direct mode for music listening.

I will admit being biased against too much processing. I have heard some interesting results from surround modes, but to me, has never seemed consistently natural sounding with different music recordings.

Ww

 

[Ridire Fáin]

Accurately measuring loudspeaker performance is pretty much a universal problem among most loudspeaker manufacturers and ALL AV review publications. Harman on the other hand under the guidance of Dr. Floyd Toole and Sean Olive have developed a purely scientific methodology that actually allows them to objectively measure and predict listener preference. This article explores the measurements while also providing the secret formula of the characteristics necessary for producing loudspeakers that score highly in subjective listening tests. This is a must read with a call to action to your favorite loudspeaker company to give us more useful information about their products.

Update (6/11/15): I added a poll question so please cast your vote and give us a reason behind your choice.

Read: How To Objectively Measure Loudspeaker Performance

Measurements can indicate the soundness of the engineering principles involved, quality of the materials, and confirm how well the design is thought out. In most cases it is a smoking gun that a product will sound good but IMHO they are not always a predictor that it actually will. They only instrument that can asses that, is the listener.

There is just too much subjective bias, psycho-acoustic affect, and personal taste when it comes to how good transducers sound. I also do not think we fully understand how both the biological (Body) and psychological (Mind) science works and then comes together within our brains. Sure, we have a grasp of the physics, physiology, and anatomy of the working parts. But what one actually perceives as good sound really differs from one person to another. This is something I believe you cannot measure.

 

[03kbredfire]

I watched this video some time back 12-18 months and was very impressed with the whole presentation. It just made sense and dispensed with flowery terms and stuck with reason and facts. I became very impressed with the M2. I would like to listen to these at some point to hear the truth. All else can be compared to the truth.

 

[shkumar4963]

Deleted

Sent from my SAMSUNG-SM-G930A using Tapatalk

 

[shkumar4963]

I voted who cares because there should be another option... along the lines of a speakers measurements that fit your subjective wants or needs... ie best match for your hearing.

Measurements and tests from reputable 3rdparty sources don't lie about a speaker's performance. It is what it is and physics cannot be argued with. Where the issue lies is the brain's interpretation of what we hear. Hearing is very subjective while measurements are objective and absolute. However audiophiles claim the following,

If it sounds good and measures bad you're measuring the wrong thing. - old audiophile expression
If it sounds bad but measures good you're also measuring the wrong thing. - old audiophile expression

The above old audiophile statements that I have quoted come out of ignorance by the audiophile community as a whole and shows a complete misunderstanding in the difference between objective measurements and subjective interpretation. Subjective interpretation does not mean listening only. It also infers other biases such as our moods, visual stimulation, etc. How many of us sat down and listened to our systems listening to the same source of music and on different days and have reached different opinions of our system? Its not the system that performed differently. Its our subjective interpretation of what we heard at that time that has influenced what he heard.

One other point I would like to make is be careful what you read into subjective experiences of professional reviewers. If two different speaker makes have very similar curves and are reviewed in the same room by the same reviewer but the reviewer reaches two totally different conclusions, then I would flag that review as suspect and question the reviewer's state of mind during the review and write up process. I would expect the conclusions to be very similar and consistent based on those curves.

You wrote: "
If it sounds good and measures bad you're measuring the wrong thing. - old audiophile expression
If it sounds bad but measures good you're also measuring the wrong thing. - old audiophile expression"

The point is that it does not. Numerous experiments have shown that if it sounds good it WILL measure good and if it measures good, it will sound good. Everything else is what they call "snake oil".

Sent from my SAMSUNG-SM-G930A using Tapatalk

 

[3db]

You wrote: "
If it sounds good and measures bad you're measuring the wrong thing. - old audiophile expression
If it sounds bad but measures good you're also measuring the wrong thing. - old audiophile expression"

The point is that it does not. Numerous experiments have shown that if it sounds good it WILL measure good and if it measures good, it will sound good. Everything else is what they call "snake oil".

Sent from my SAMSUNG-SM-G930A using Tapatalk

Thank you for agreeing with me. I may be old but I'm NOT an audiophile

 

[Mikado463]

I may be old but I'm NOT an audiophile

not even the first paragraph ? ............ https://en.wikipedia.org/wiki/Audiophile

 

[3db]

not even the first paragraph ? ............ https://en.wikipedia.org/wiki/Audiophile

not even the first paragraph ? ............ https://en.wikipedia.org/wiki/Audiophile

If you reread, I said audiophiles claim ..... I dont run on that side of the tracks.

 

[Mikado463]

I dont run on that side of the tracks.

all good, just stay off the tracks, you don't want to get run over by the train ...........

 

[John Parks]

not even the first paragraph ? ............ https://en.wikipedia.org/wiki/Audiophile

That is a good, if simplistic take on things. As for me personally: sentence (1) agreed; sentence (2) disagree, except for the acoustics part, sentence (3) yup.

 

[Speedskater]

If we consider:

• a number of listeners
• a number of musical genres
• a number of recording techniques
• a number of listening rooms

Then the Toole, Olive, Harman measurements will be a vary good predictor.

 

[3db]

all good, just stay off the tracks, you don't want to get run over by the train ...........

Sounds like a threat..

 

[Mikado463]

Sounds like a threat..

nah, just some sage advise !

 

[highfigh]

Now if they could only come up with one that worked on women...

If anyone ever does and it turns out to be accurate, we'll never hear about it because, well, you know, women have a kind of telepathic network and they'll all know about it as soon as it's active. By "we'll never hear about it", I really mean "We'll never hear from the inventor".

In the mean time, there's this-

https://etherealmind.com/wp-content/uploads/2011/02/man-woman-knon.jpeg

 

[Ossidian]

While the report below refers to amplifiers it shows that measurements do not necessarily correlate with musicality.

https://linearaudio.nl/sites/linearaudio.net/files/Hirata test engineering report 1981.pdf

 

[Ossidian]

You wrote: "
If it sounds good and measures bad you're measuring the wrong thing. - old audiophile expression
If it sounds bad but measures good you're also measuring the wrong thing. - old audiophile expression"

The point is that it does not. Numerous experiments have shown that if it sounds good it WILL measure good and if it measures good, it will sound good. Everything else is what they call "snake oil".

Sent from my SAMSUNG-SM-G930A using Tapatalk

Perhaps you might enjoy reading the report below. Measurements do not necessarily correlate with good sound.

https://linearaudio.nl/sites/linearaudio.net/files/Hirata test engineering report 1981.pdf

 

[Bucknekked]

Perhaps you might enjoy reading the report below. Measurements do not necessarily correlate with good sound.

https://linearaudio.nl/sites/linearaudio.net/files/Hirata test engineering report 1981.pdf

Ossidian
I took at look at the report you quoted just to see if this was another case of someone claiming a conclusion that wasn't actually expressed in the research paper. That's what I see here. The conclusions of the research paper do not express that measurements don't correlate with good sound. The conclusions of the research paper laud the method of testing proposed in the April 1981 paper. Since its 2019, some 38 years later, I'm assured that there have been new methods developed and tested and new measurement systems too. A lot happens in 38 years. It was still a good research paper and I'm sure if we recreated the tests, they would still pan out today.

The basic premise of this thread is to pose the question "can measurement predict subjective performance?". That question has been thoroughly discussed, sliced n diced, and I think pretty well answered in the affirmative.
Measurements, done well, can predict from a group which speakers will sound better than others. Toole and others have demonstrated it to be true.

What measurements won't be able to do is handle the end users subjective reactions on any given day. We, the end users are the least reliable and least dependable part of the equation. I myself demonstrate that variability on a regular basis. This week in my listening sessions, the music has been stellar. Outstanding. Nothing out of sorts at all. Every now and again however I will have listening sessions where the opposite is true. Nothing seems to sound very good. Nothing is spectacular. The music sounds mundane. Is it the fault of my hardware? Did my speakers spoil over the weekend? Nope. Its me the constantly variable meatbag that is the culprit.

Can measurements do their job? Yes. Can they predict what joe music user will think or feel about a system? Probably not. The end user is too variable a mess. You can't blame the mess on measurements inability to figure out you need a burrito and a beer before you listen to your music.