The Ethics of Product Review

A

av_phile

Senior Audioholic
Here's a quote from one of Gene's responses on the Z9 thread. And according to the administrators, I should just post my questions here. Highlight is mine.

The RX-Z9 will handle NTSC and PAL, not sure about the variants you mentioned. I will ask Yamaha after they look over my review. There will likely be a review addendum after my review is posted to handle everyones questions, or we may just feature it on this forum if it remains highly active.
I am curious. Why should an INDEPENDENT reviewer allow his review to be looked over by the manufacturer of the product under test?

Having worked with film critics and consumer advocacy groups in the past on the side, I have never encountered a film reviewer submitting his review of the film to its director prior to publication. That would be like showing the pupils the result of their exams before finally putting in the grade. It is my undestanding that a review or test is supposed to validate the manufacturer's claims on the product. So it would be unethical for the manufacturer to have even a hint of the review prior to publication.
 
gene

gene

Audioholics Master Chief
Administrator
As usual AV_Phile you take my commentary or others commentary out of context. Yes we do allow manufacturers to look over the technical aspects our reviews before posting to ensure we did not incorrectly state things about the product(s) in question. This is a professional courtesy we extend to manufacturers as do most credible product review websites and magazines. This is especially important to us because we like to provide indepth technical reviews/analysis unlike much of our competition that simply do cursory one pagers and want to make sure we have our facts straight. Its often difficult to get all the facts on products when you don't have schematics, firmware, etc to confirm your findings or suspicions and the manufacturers can be of great help in this regard.

This is also a policy we made explicitly clear and if you spent more time reading the review section of our site rather than obsessing over all channels power driven tests you would know this. If you cannot see the merit or logic in this, I don't know what to tell you. We are neither film critics or part of a consumer advocacy group so I cannot comment on their policies.

If you don't like or agree with our policy, you are more then welcome to not participate in our free forums or read our articles that we offer the public for free. It's weird that you are like this, especially since you spend more time on our forums than almost any of our members. Perhaps you just enjoy arguing with people. Unfortunately I don't have the time or desire to do so.

Most of the people on this site spend countless hours contributing out of shear joy and love for the hobby. You discredit their efforts and their work by your continuous compulsion to point fingers of foul play at work here both in these forums and in private messages you have been sending myself and my staff over the past week.

It's your choice to use this site, enjoy the forums, and participate. You usually offer very good contribution here, but quite frankly if you continue down this path, don't let the door hit your *** on the way out! There I vented my steam, nuff said :mad:
 
S

shlomore

Enthusiast
Keep The Good Work Gene !!!!

as a member here i would like to say that not only this site is so professional,gene which doing great job is a very professional in what we all like in here..
thank you so much gene and everyone in this great site !!!













gene said:
As usual AV_Phile you take my commentary or others commentary out of context. Yes we do allow manufacturers to look over the technical aspects our reviews before posting to ensure we did not incorrectly state things about the product(s) in question. This is a professional courtesy we extend to manufacturers as do most credible product review websites and magazines. This is especially important to us because we like to provide indepth technical reviews/analysis unlike much of our competition that simply do cursory one pagers and want to make sure we have our facts straight. Its often difficult to get all the facts on products when you don't have schematics, firmware, etc to confirm your findings or suspicions and the manufacturers can be of great help in this regard.

This is also a policy we made explicitly clear and if you spent more time reading the review section of our site rather than obsessing over all channels power driven tests you would know this. If you cannot see the merit or logic in this, I don't know what to tell you. We are neither film critics or part of a consumer advocacy group so I cannot comment on their policies.

If you don't like or agree with our policy, you are more then welcome to not participate in our free forums or read our articles that we offer the public for free. It's weird that you are like this, especially since you spend more time on our forums than almost any of our members. Perhaps you just enjoy arguing with people. Unfortunately I don't have the time or desire to do so.

Most of the people on this site spend countless hours contributing out of shear joy and love for the hobby. You discredit their efforts and their work by your continuous compulsion to point fingers of foul play at work here both in these forums and in private messages you have been sending myself and my staff over the past week.

It's your choice to use this site, enjoy the forums, and participate. You usually offer very good contribution here, but quite frankly if you continue down this path, don't let the door hit your *** on the way out! There I vented my steam, nuff said :mad:
 
Yamahaluver

Yamahaluver

Audioholic General
av_phille,

The crux of your problem doesn’t really lie with Gene’s review but your strong and apparent bias towards anything Japanese, especially any product bearing the Yamaha name. If we were to analyze all your previous posts, one can easily come to the conclusion that either you suffered a real sad experience with Japanese products or Yamaha as you have not even a single good word for them. It is perfectly fine to have your biases and opinions, however strong that may be as we as humans are entitled to that freedom but imposing them on others or trying to shape it to your personal preferences reeks of dogmatism which was quite prevalent in the Soviet Union.

You would rather that Gene rants and raves about your favorite snob appeal names like Rotel etc. rather than dare to mention the dreaded Yamaha word. The reason Gene sent out the review for perusal by the manufacturer shows of AH’s concept of fair play which is what drew to me for the first time and I continue to do so as long as Gene and others allow me. This gives the manufacturers to point out any technical anomalies in the review or a chance for the manufacturer to clarify certain points, I along with others would have it no different way and if this irks you, there are so many other forums catering to snob appeal, non Japanese brands and as a matter of fact, the mere mention of anything Japanese on those particular forums would get you barred and detested to say the least. Why do you still hang around here and get really frustrated is beyond my comprehension. Sadly it seems that the editorials as well as the general atmosphere of the forum doesn’t suit your disposition, one thing you should get by now. We all value our opinions however slanted, stupid they may be and to change them would be a totally futile exercise on your part.

Sadly you remind us of an ex-member who is thankfully not here anymore as to him, anything from the east has no right to be in the audio forum. You are slightly different as your concepts are much clearer than him, only that your bias clouds your position from time to time. Learn to live with that and we all would have a good time.
 
S

steve

Audioholic
Japanese vs American

Greetings AVPhille,

We are sorry you feel that we only favor Japanese products. But I don't think we have any special favoritism to Japanese designed and/or made products. In fact, we did an in depth article on the Parasound Halo C 1 and C 2 and the review was very positive. We encourge you and others to read this review and see that we did our best to find ways to work around some of the missing elements on these aging products because we felt they were worthwhile products to consider in a vast, rapidly changing, highly competitive marketplace.

Parasound Halo C 1 and C 2 Review

If memory serves, the only non-Japanese Processor we hit pretty hard was the Sunfire Theater Grand III. Our review of that product was indeed critical, but given it's pricetag, we simply felt it had too many shortcomings to be competitive. You're welcomed to read that review as well.

Sunfire Theater Grand III

One thing you may want to consider is that we do try to be objective when possible while also supporting our subjective comments with facts and/or experience. This is the very reason why we provide measurements when possible and why we created an AV Processor/Receiver Checklist.

AV Processor/Receiver Checklist

This list was completed after careful consideration and thought combined with input from many people in the field from companies in America, Sweeden, Japan and other places. We tried to compile this list by listing product offerings and assigning them a logical score based on what we felt was critical, important, or bonus features. We then use this table to score the products under review. I'm not sure how much more objective we can be beyond this as all Processor or Receivers are placed under the same scoring process. In fact, this is an adaptive score card as each year there new offerings that are added to the list.

As for our measurements, we have a growing list of test equipment that we use in speaker analysis, cable analysis, video analysis, amplifier analysis and other analysis of many products. This growing list of equipment can be viewed in our evolving table.

Audioholics Test Equipment

We even completed several training classes including ISF, HAA, RF, and several others, all in an effort to gain more knowledge to provide better and more complete reviews.

Now grant it reviews can be subjective too, including ours. That's just the nature of a review and why we are not just a test lab. We do our best to support our subjective writings with objective facts as often as possible and as best we can. But even magazine publishers author articles that are subjective and may be considered biased by some. I don't think there's a steadfast solution to this misconception, but we're willing to listen. Much of our subjective views are based on our experience with tons of products, and typically not from the seat of our pants.

Thanks for continuing to visit Audioholics and discussing in our forums. We do hope you continue to provide valuable input and comments. But we ask that you consider not being as abrasive to us when you disagree as we find that it's counter productive. We always value and welcome constructive criticism as it helps us improve our site and services. There are time however, that we get a bit taken back by argumentive comments in place of critical evaluations. Perhaps this is due to the fact that we are all Engineers and not Philosophers.

We do wish for you to continue to provide valuable feedback and comments, but please be a bit more tactful in your approach as it will go a long way with us.
 
D

Dan Banquer

Full Audioholic
Reviews

As one who has had his company (www.redesignsaudio.com) reviewed by a number of different magazines I can tell you that it is not uncommon for reviewers to show the review before it is published for technical editing/comments. Since many reviewers are technically illiterate they need to get certain facts straight. I do not get to comment on any of their subjective impressions.
This web site is anything but technically illiterate and the purpose of showing the review on a technical level is to allow the company to address the problems the staff here has found. Note that many of the products reviewed here may need some software changes which the manufacturer was initially not aware of. This results in progress, not animosity if the parties involved are rational.
Dan Banquer
R.E. Designs
 
RLA

RLA

Audioholic Chief
Hi All
I see nothing wrong with sending the review to Yamaha before publication
There is a distinct difference between reviewing a performance (Movie)
and reviewing a very sophisticated piece of electronic equipment
proof reading for technical correction or user/reviewer set up would be the main reason for the proof read
 
W

Westrock2000

Junior Audioholic
It also gives the reviewer time (or the option) to include a rebutle from the manufacturer. Whether that rebutle be simply thanking the reviewer for a positive review or trying to explain aspects that were frowned upon. Although not particulary common in reviews, I think its a good addition because it allows us to see how the manufacturer feels and/or their attitude.


I thought this was going to be something like posting a review based on second hand experience, or a complete lack of faith in a product for the wrong reasons. Like saying that a speaker will suck simply becuase it has a vented bass driver, or that one reciever is better than the other based on wattage.....thats unethical reviewing.
 
W

Westrock2000

Junior Audioholic
Heh, one thing that gets me about some reviews in magazines is that of speakers. Have you ever noticed that sometimes the domes on the tweaters are pushed in on the sample picture? Whats up with that? Are reviewers that careless? Did they recieve them like that? It just looks plain bad for everybody. Becuase then you wonder if they listened to them like that (although many claim it affects are nill), or if the company has bad Quality Control.
 
P

Physique

Audiophyte
Dear Av Phile,

As an engineer who used to head the testing in the Technology department of a top Networks company, I can tell you that the technology industry is a very different one from the film industry. Part of my job was to study different Internet connection technologies and develop standardized methods of testing for each one. Another part of my job involved reviewing each of these technologies. In both instances, I always contacted the manufacturer of the product/technology under test after my test report was finilized as this accomplishes two purposes:

1) It gives the manufacturer an opportunity to validate the test setup. Sometimes faulty logic in test method or incorrect firmware/software loaded in the product may have been the reason why a result did not come out as expected.
2) It provides the manufacturer with good feedback on their product if the findings are indeed valid.

There is nothing unethical about this. As a matter of fact, as Gene said himself, this is a professional courtesy if nothing else, and serves to preserve the validity of the review.

Comparing the technology industry with the film industry is like comparing an apple to an orange. A film review, does not need to be validated as it is subjective in nature. While it can be argued that parts of a technology review can be subjective as well, if you wish to make a strong case for your opinions, your test setup and technology/product facts better be straight or else your case will fall faster than water at the Niagara Falls. Ergo, the reason for a peer review by the manufacturer of the product in question. What you are doing here is just setting up a classic strawman's argument and at such a contest you stand at a disadvantage.

Best of Health,
Hugo A. Rivera
President
HR Fitness Inc.
http://www.hrfit.net
 
zipper

zipper

Full Audioholic
I enjoy reading the reviews on this site. I find that they are objective,in-depth,& professional. When there are criticisms of a product,they are done tactfully & without malice. Thank you Gene,Steve, & Clint. You provide an avenue for less experienced audio enthusiasts such as myself to gather ideas & information without having to wade through a ton of BS or egotism.
Just about all the regulars here have more experience & knowledge of products than I do,so I thank them for sharing.This includes you AV_phile. But I don't understand why you're so negative. Yamahaluver is obviously pro-Yamaha. Just let it be. You could change your name to Roteluver & pump Rotel. I got no problem with that. But you seem to ***** for the sake of *****ing. IMHO,you have much to offer on a site like this but remember that,like the rest of us,we are guests here.We don't own it. Lets not denigrate it or it's owners for trying to provide a free service & platform for sharing.
 
Rip Van Woofer

Rip Van Woofer

Audioholic General
Okay, let's get down to the REAL question:

How much are Gene's reviews influenced by his brother Steve's homemade mead?? :eek:
 
D

Dan Banquer

Full Audioholic
Review Ethics

"How much are Gene's reviews influenced by his brother Steve's homemade mead?? "
Good Question! Enquiring minds want to know! :D
Film at 11?
d.b.
 
gene

gene

Audioholics Master Chief
Administrator
It's not the wine thats getting to me, its sleep deprivation. Damn I need a vacation or more hired help :eek:
 
Rip Van Woofer

Rip Van Woofer

Audioholic General
You know what they say about self-employment and small business, Gene: you work 16 hours a day for yourself so you won't have to work 8 hours a day for someone else!
 
gene

gene

Audioholics Master Chief
Administrator
Or you work 8hrs/day for someone else and 16hrs/day for yourself at the same time. I am hoping to eliminate the prior very soon! I don't mind working around the clock for myself, especially since I love doing this :)
 
Yamahaluver

Yamahaluver

Audioholic General
gene said:
Or you work 8hrs/day for someone else and 16hrs/day for yourself at the same time. I am hoping to eliminate the prior very soon! I don't mind working around the clock for myself, especially since I love doing this :)
Gave up working for others long time back, most unrewarding and constant nagging made me switch to self employment and I have never looked back.
 
gene

gene

Audioholics Master Chief
Administrator
Yamahaluver; I hear ya. I don't mind the extra work involved in working for yourself and the many hats to have to wear to be successful in it. It beats sitting in a Dilbert cube doing work you hate and answering to someone you would rather have no affiliations with whatsoever. I love working at my own pace and setting my own priorties and being in total control of my own future. Although more risky, it is much more enjoyable to me.
 
D

Dan Banquer

Full Audioholic
Product Reviews

You mean Gene has some software weenie measure cables? :eek: OH boy we're in real trouble here folks. Does the Wayne kerr Analyzer have a fat screen? :rolleyes:
d.b.
 

newsletter
  • RBHsound.com
  • BlueJeansCable.com
  • SVS Sound Subwoofers
  • Experience the Martin Logan Montis
Top