Equipment Reviewing Standards

[FozzieT]

Here is an idea as a long-time audiophile, but new poster. Isn't there any possible way that reviewers can come up with a list of, say 5 songs to review equipment with?

They can use additional songs, but all reviews would at least include these 5 songs.

The 5 songs would be well-recorded, well-mastered songs with which many people are already familiar. They would span a variety of music genres. Each song would demonstrate a variety of aspects such as soundstage width and depth, instrument imaging, transient response, and so on.

One song might be, for example, Pink Floyd's Welcome to the Machine. It is a well recorded song, most people have heard it, and it displays a variety of the different aspects by which to judge audio equipment, such as imaging, transient response, bass depth, etc.

Wouldn't it be easier to understand a speaker system review, if you always had a frame of reference by which to compare the sound? "On "X" System, I found the beginning machine noises to be overly bright, as compared to "Y" System." Most people have a frame of reference to that song, so they can understand the comparison.

It wouldn't necessarily need to be THAT particular song, but at least something more popular than "Obscure Artist, and only then, on one particularly narrowly-released vinyl album". "That I've never listened to on any other system."

Choose one song from different genres: Classical, Jazz, Rock, Electronic, Country/Folk... Okay, I know there are thousands more genres. And reviewers could use them. But if they had at least 5 standard songs, it would make their reviews a lot more meaningful.

But, alas, the less objectivity in critical reviews, the better.

 

[crossedover]

As for subjective reviews sure. However science has proven that the mind can't remember differences for more than a very short time. If choosing music for critical review or auditions that would help over come some of that, then choose unamplified acoustic music with just a couple of instruments and a voice. From sensory to long-term memory: evide - PubMed Mobile
http://www.google.com/url?q=http://web.missouri.edu/~cowann/docs/articles/in%20press/now%20out/Winkler%20%26%20Cowan%20Exptl%20Psychol%202004%20in%20press.pdf&sa=U&ei=_tpDU7-WHMmvsQTD8oH4Bg&ved=0CBoQFjAD&usg=AFQjCNHzf66hFA7JWfhb1Elgvy4HRYQEKA

 

[cpp]

I think a huge issue would be getting all reviewers to agree on which 5 songs to use regardless of genres. Then the issue becomes, their amps or receivers , source equipment and speakers used to test a certain device/speaker and not to mention their listening room, listener distance was the room treated or not.. Nothing will be easy, even these reviewers age can come into question not to mention their hearing abilities. It could be "total madness".

 

[defmoot]

It could be "total madness".

Hysterical. Early favorite for thread winner...

HST, I kind of like the OP's idea, at least in theory. But I wanna pick the songs.

 

[ReUpRo]

Picking 5-10 tracks that cater to everyone's listening tastes is not possible. Maybe, 5-10 tracks from every genre would be a more reasonable endeavor. Even then, there are too many variables like, room, human,

Interesting thought experiment though. IMO, measurements and testing could be the even playing field you're looking for. Even then, every publisher has their own benchmark. It would be an easier task to specify a paradigm to become the de facto standard. For example, LEDE for room design, Audioholics Amp measurement protocol, Bass-data for subs, etc.

 

[JMJVK]

Sound is subjective...

Another subjective hobby of mine is astronomy. After years spent in the same forums, with the same guys, I've come to know what these guy's opinions mean and what I should expect to feel about the same gear, should I purchase it. Audio gear is the same.

Sound systems cannot be de-subjectivized. Numbers and graphs can be used, but can't convey the experience. The best you can hope for, is understanding your reviewer's idiosyncrasies and biases, relative to your own, and use that knowledge to decode what to expect and if it suits your taste. The choice of tracks used is irrelevant, as reviewers know how they like their tracks to sound like.

 

[ReUpRo]

Another subjective hobby of mine is astronomy. After years spent in the same forums, with the same guys, I've come to know what these guy's opinions mean and what I should expect to feel about the same gear, should I purchase it. Audio gear is the same.

True, but, a Type 1a Supernova brightness is not subjective. Here we're trying to define what Type 1a would mean .

 

[Adam]

True, but, a Type 1a Supernova brightness is not subjective. Here we're trying to define what Type 1a would mean .

The brightness may not be subjective if measured by precision instruments...but that's not what we're talking about here. To have different people listen to the same songs on different systems - to me, it doesn't matter if the songs are the same. It's questionable if the same person could be moderately objective when listening to the same songs days/weeks/months/years apart with different gear...but I think it's pretty clear that it's not apples to apples once you have different people doing it. My two cents, anyway.

 

[PENG]

If you believe in go by your ears, then pick your own tracks and you are done. If you don't, then I would say it is better to come up with standards measurements using standard equipment, methodology and that such measurements be done by people who know what they are doing. I have to say this again, we don't get to the moon by trial and error, hearing and seeing. You go and you land right there, the only thing you can count on is applied science and engineering principles and of course measurements taken to verify the gear involved are performing what they are designed for.

Audio precision/fidelity should work the same way, at least for audio amplifiers, manufacturers should be able to design and build one base on good science/engineering know how, verify performance standards are met by measurements, and they should just amplifier the signal without introducing any sort of audible distortions. There is probably a need to establish some international performance standards so reputable manufacturers have to meet them and measurements can be taken to verify the same. For speakers, same principles apply but will be much more complicated in practice for obvious reasons.

 

[JohnnieB]

If you make a standard that every input has to be the same, every outcome has to also meet a specific standard, and everyone should get the same thing in the end so we would know what to expect....we would all be driving camrys. But you get to pick the color. I get what the op is saying, but I don't want my system to sound like someone elses. Yes knowing what to expect sight unseen or sound unheard, may make finding your match faster or easier. In the end, I think it's the not knowing, that makes this hobby great. JMO

 

[cpp]

.but I think it's pretty clear that it's not apples to apples once you have different people doing it. My two cents, anyway.

That's true as there are different varieties of apples... which still means the people whom are the reviewers all will be subject to ridicule not to mention their hearing will be questioned ....

JohnnieB ; If you make a standard that every input has to be the same, every outcome has to also meet a specific standard, and everyone should get the same thing in the end so we would know what to expect....we would all be driving camrys.

well said. Not to mention since music and equipment manufacturing is global this would have to be an international standard.

 

[Adam]

That's true as there are different varieties of apples...

Is no saying sacred anymore?

J/K.

 

[cpp]

Is no saying sacred anymore?

J/K.

so true, so very true

 

[zieglj01]

Here is an idea as a long-time audiophile, but new poster. Isn't there any possible way that reviewers can come up with a list of, say 5 songs to review equipment with?

They can use additional songs, but all reviews would at least include these 5 songs.

Did you ever read this?
The Official $3,000 Speaker Evaluation / Home Audition Event - Home Theater Forum and Systems - HomeTheaterShack.com

 

[JMJVK]

True, but, a Type 1a Supernova brightness is not subjective. Here we're trying to define what Type 1a would mean .

The objects and classifications are no what's subjective in amateur astronomy. It's the favoured optics and merits thereof, which are discussed "Ad nauseam". This discussion is rendered even more obtuse when you start adding the possibility of people preferring one or the other depending on the target type, such as a galaxy, planet, planetary nebula or diffuse nebula.



With sound, you have subjective preferences of music types, and of "sound" itself (boomy, neutral or clear) and you can add the subjectivity of what one judges as being "boomy, neutral or clear", on top of that. To add yet more mayhem, you have room configuration, construction and finishing materials and the furniture to boot.

Some even judge kit using heavily synthetic and/or distorted types music. I like Metallica, AC-DC and Pink Floyd, but I believe that to hear whether music sounds correctly reproduced or not on any type of audio equipment, acoustic quartet music, and music leaning heavy on strings, flutes and pianos offer the best reference. As people are reading these lines, I'm sure a bunch of them are already inwardly protesting, and can't wait to finish readin so they can "correct" or "re-align" my opinion.



Subjectivity of human senses just can't be eliminated.

I think it's pretty clear that it's not apples to apples once you have different people doing it. My two cents, anyway.

I agree 100% on that statement.