Why are objective listening tests not used for Audioholics reciever tests?

U

Unregistered

Guest
I thought Audioholics was about using scientific testing to determine differences between audio products?

If so, then why do I see such suggestions on the reciever buying guide to compare recievers using such teqniques as "Turn the volume control up about half way and balance all channels using the test tone. Now walk around the room and listen for airplane noises. Assuming you are not near an airport, you should not hear any. If you do, than this is a good indication that the receiver has a noisy preamp section. In a seated position midway between all speakers, listen for excessive background noise."
http://www.audioholics.com/techtips/buyingguides/avhardware/receivers.php

That doesnt sound objective to me. How do you know you would even hear that noise when listening to music or movies in a normal environment?

"You should compare the fidelity of sound of the internal DAC's with that of the CD/DVD player you are using in your system. When switching from analog to digital inputs on the receiver, you should notice similar or better fidelity."
I believe that all resonable quality dacs in recievers and dvd players should sound the same. Why hasnt audioholics compared recievers, dacs, or amplifiers with scientific scrutiny? Subjective listening tests IMO are a poor way of determining anything due to BIASES.

Under his two channel section, he asks users to make sure that sound is free from "excessive boosting or attenuation". Yet if Audioholics would MEASURE the output of the amplifiers, they could easily tell that the frequency response of MOST amplifiers is completely flat within 1 decibel.

Would Audioholics agree that biases can be introduced when the subject knows what device is playing (ala cables)? So why the heck don't you use the same testing when comparing other audio devices? Come on guys....
 
gene

gene

Audioholics Master Chief
Administrator
My first question back to you, is why are you an anonymous poster? You raise valid concerns, wouldn't you prefer to give credit to your name? Ok thats two questions.

The article you reference is over 4 years old and definately needs updating. Until you pointed me to it, I didn't even realize it was still on the site. Most of the comments (ie. airplane noises,etc) were meant as a joke and to spot check the noise floor of the product with no signal applied.

Yes things should be measured, such as frequency response, distortion, etc, provided that you have access to the proper test gear. We are still assembling the necessary test gear for such measurements. We are fortunate to have a $30K Magnetics Analyzer at our disposal so we could do detailed cable measurements and analysis. We also purchased some of the best test gear for measuring speakers, room acoustics, and video display. These measurements are integrated into many of our reviews. We also have a dedicated page which lists our test gear. Note this doesn't include the Sencore Vector scope we just scored.

List of our Current Test Gear

We are working with very big and well recognized companies to acquire audio analyzers to measure frequency response, distortion, etc. We are also working on standards with these manufacturers to make these measurements which are mostly lacking in this industry.

This is why we formed the Home Theater Alliance (HTA) at:
http://www.hometheateralliance.com

All of our senior staff are ISF and HAA certified as well.

Stay tuned, we are just getting warmed up and things are gonna get interesting :)
 
U

Unregistered

Guest
Thanks for the quick reply Gene. I am sorry I came off so harsh at first.

The main point of my post wasnt over your measurement techniques. What I really am concerned about is that Audioholics does not test using double blind tests? Thats why I brought those things up. You seem very knowledgeable about the scientific process. Why arent scientific listening tests done then when asked to compare equipment? It is ok for us to say that there is no "sonic signature" for a cable, but what about amps, dacs, and other electronics? Im almost positive that I am not the only Audioholics reader that believes when double blind tests between amplifiers (working within their linear limits and precisely matched in volume) or dacs that people can not distinguish the difference in many cases. How can you explain these results: http://www.pcavtech.com/abx/abx_cd.htm You use the scientific method, do you see any problems with his comparison techniques? Or how about this one: http://www.pcavtech.com/abx/abx_pwr.htm? (no i am not affiliated with pcavtech :)

I just simply think that in the subjective section of your reviews, you should use OBJECTIVE blind testing first to even see if there is a difference at all. I am almost sure this post won't change your opinon, but in the small chance that it may have, I feel that this has been well worth my time :)
 
U

Unregistered

Guest
Oh and I certainly applaud your use of quality measurement equipment. But even such publications as Stereophile use objective measurements, yet come to the conclusion that there are noticeable sonic signatures between audio cables. From what I read on Audioholics, you do not believe the same things. Sure we can measure it, but can we hear it??? Thats my point.

The reason I was so upset in my first post was because I have read many of your cable articles in the past and enjoyed them thoroughly. There are few objective publications on the net. Yet someone on another forum posted a link to that recicever article, and in my opinion it completely lacks what you call "finding the truth in audio" in some of its aspects :(
 
J

Jerry Parker

Audiophyte
Hi, sorry I didnt register before, but I am registered now
 
gene

gene

Audioholics Master Chief
Administrator
Im almost positive that I am not the only Audioholics reader that believes when double blind tests between amplifiers (working within their linear limits and precisely matched in volume) or dacs that people can not distinguish the difference in many cases.
Jerry, first off thanks for registering.

I agree with you 100% here. However, we rarely do amplifier face off reviews for several reasons:
1) Too time consuming
2) Most manufacturers won't send gear if they know you are doing a face off
3) Usually doesn't make for good reading material. Most consumers want to know the details of the product in question, not a glance over of a bunch of products. Most of our reviews are over 15 pages in PDF format!
4) It is an enormous effort to do this right, have all variables under control, etc, and people would still not likely be happy with the results.

Stay tuned however, as in the coming months we will be in a better position to do more objective testing on gear to supplement our thorough reviews.
But even such publications as Stereophile use objective measurements, yet come to the conclusion that there are noticeable sonic signatures between audio cables. From what I read on Audioholics, you do not believe the same things.
Actually to my knowledge, Stereophile has never measured cables, especially to the level of detail that we do. I actually measure the cable metrics, model them in sophisticated programs and verify it with mathematics and show correlations between all three!

Sure we can measure it, but can we hear it??? Thats my point.
While that may be true for hardware (amps,CD players, Speakers), this is simply NOT the case for cables. There is no magic to cables. Nothing undefined. It is an exact science. The test gear I use is on the order of 1000's of times more accurate and consistant than the human ear.

The correct statement should be, just because you can measure a difference in cables, doesn't mean that you can hear it!

With that, I have shown why some cables do sound different, which is a result of their measurements and how they interact with the interface equipment.

BTW, if you find any other articles that are grossly outdated please email me privately and I will take a look. Some of our old stuff either needs updates or the trash can ;)
 
J

Jerry Parker

Audiophyte
gene said:
Jerry, first off thanks for registering.

I agree with you 100% here. However, we rarely do amplifier face off reviews for several reasons:
1) Too time consuming
2) Most manufacturers won't send gear if they know you are doing a face off
3) Usually doesn't make for good reading material. Most consumers want to know the details of the product in question, not a glance over of a bunch of products. Most of our reviews are over 15 pages in PDF format!
4) It is an enormous effort to do this right, have all variables under control, etc, and people would still not likely be happy with the results.

Stay tuned however, as in the coming months we will be in a better position to do more objective testing on gear to supplement our thorough reviews.


Actually to my knowledge, Stereophile has never measured cables, especially to the level of detail that we do. I actually measure the cable metrics, model them in sophisticated programs and verify it with mathematics and show correlations between all three!



While that may be true for hardware (amps,CD players, Speakers), this is simply NOT the case for cables. There is no magic to cables. Nothing undefined. It is an exact science. The test gear I use is on the order of 1000's of times more accurate and consistant than the human ear.

The correct statement should be, just because you can measure a difference in cables, doesn't mean that you can hear it!

With that, I have shown why some cables do sound different, which is a result of their measurements and how they interact with the interface equipment.

BTW, if you find any other articles that are grossly outdated please email me privately and I will take a look. Some of our old stuff either needs updates or the trash can ;)
That's understandable, I guess I never thought about the fact that the manufacturers do send you the stuff for free to review.

And lord only knows how people would respond to a DBT. I know many forums have banned DBT discussion (usually the cable section of the forum) because many dont agree that DBT's give any valid conclusions.

My mistake on the Stereophile part. I confused the measurements of the speakers and amplifiers they did and thought they did them on cables too.

I must have poorly worded my post, I did mean to say "just because you can measure a difference in cables, doesn't mean that you can hear it!" I certainly agree that amps, dacs, cables and other things can DEFINATLY measure differently, but when subject to bias controlled listening tests, the participants cant reliably detect a difference.

And yes, I too agree that well designed cables CAN sound different, but that is the exception and not the rule, wouldnt you agree? (cables with fat network box in the middle excluded from "well designed" IMO) ;)
 
gene

gene

Audioholics Master Chief
Administrator
Jerry;

And yes, I too agree that well designed cables CAN sound different
Actually I disagree here. Well designed cables should be indistinguisable sonically. The problem is just how well is well :D
 
RLA

RLA

Audioholic Chief
Hi All
One of the first things I learned back in the 70's (Yeah I'm getting to be a geezer) :p Is there is no such thing as objective listening tests when
Reviewing or demoing equipment The only way an objective listening test could be done is if the reviewer never sees or knows anything about the equipment he is about to review. I guess you would call it a sequestered review. All one has to do is see, feel, or hear about a piece of equipment and the objectivity is gone and replaced by a pre conceived notion, It is much easier in my opinion to be objective when using raw data and scientific measurement to present the facts to the reader of the review But I suppose if you wanted objectivity in a listening review it could be done with 3 or 4
Reviewers in a blind folded listening test conducted in random order
The other thing that amuses me is so called objective reviews in Magazines
Followed by a full page ad for the product that was just reviewed what do you suppose is the percentage of bad reviews or even just plain acceptable
Reviews given to products reviewed in Stereophile? Can it be that most all the products that Stereophile reviews are stellar performers? Or could it be that the reviewer is not that objective after all?
Cheers
 
J

Jerry Parker

Audiophyte
gene said:
Jerry;



Actually I disagree here. Well designed cables should be indistinguisable sonically. The problem is just how well is well :D
LOL, I meant in terms of noise rejection. In some cases, using different cables CAN fix audible noise, wouldnt you agree? :D
 
gene

gene

Audioholics Master Chief
Administrator
Yes I do agree Jerry. Shielding is usually not needed for speaker cables, but for interconnects and video cables that is another story. FYI, one of our contributors is an industy expert on EMC and signal integrity, Mr. Henry Ott. Check out the following cable article for more information:

Spiral Shields

We also have a Cable Budget Guidelines that highlights some of the important metrics of cables. We don't allow advertising from cable companies that don't adhere to these guidelines nor do we all advertisement from cable companies that promote Cable Snake Oil. I am sure this site would be more lucrative if we did, but I would have problems sleeping at night :p
 

Latest posts

newsletter

  • RBHsound.com
  • BlueJeansCable.com
  • SVS Sound Subwoofers
  • Experience the Martin Logan Montis
Top