I see we're once again treading the great subjective/objective divide, hmm?
OK, let me address (from an objectivist viewpoint) a few points raised by both:
There are lots of perfectly valid reasons for even a hardcore objectivist to buy "rockin" high-priced gear that have nothing to do with audible differences. Things like build quality, a longer warranty, more attention from customer service if things go wrong, features, ergonomics, even intangibles like pride of ownership and styling. As I've often said, if I could afford them I'd own Bryston amps for all those reasons, and I'm as much a hardcore DBT advocate as mtry (though more diplomatic about it!)
But I know that my humble Adcom, Pioneer and other components do their basic jobs just as well as their more expensive equivalents, and I doubt they would "sound different" at matched levels.
Autosound was brought up. Now, autosound ain't my thang but I have heard that mobile gear is a lot more variable in performance than home gear, so maybe there are real audible differences between different head units, amps, and such. Maybe we're in apples vs. oranges territory here. Maybe. But I'm on shaky ground due to my own ignorance of autosound.
On modders in general: Most of the mods I've seen involve the replacement of stock components with expensive "boutique" components (fancy caps and the like). This is done on the simplistic assumption that they will "sound better". Usually there is little or no scientific or engineering rationale for this that will hold up under scrutiny. Now, it is possible that the modded item will indeed sound
different but is it really better? IMO, if gear sounds "different" there's probably something fishy going on since audio equipment should have no "sound" at all (and most modern, competently-designed gear these days doesn't, as DBT and measurement both routinely reveal), so it's probably actually a change for the worse. There is a tendency for people to assume different is better. And let's not forget the power of suggestion.
Now, there is the common assertion that cheap parts are often used due to budget/price constraints. True. But it is also true that quite often the cheaper parts will only slightly affect the measured performance, and that the difference will still be inaudible. And it is also true that using expensive parts in a given application will often simply be a waste of money imparting little or no measurable benefit; but doing so will impress the naive consumer and can be used to justify some dubious marketing claims, a higher price and greater profit margin for the manufacturer.
OTOH, there are some good reasons for using more expensive parts that have nothing to do with electrical performance or sound: things like reliability (see "build quality" above). So, for instance, a film cap in a certain application might indeed be "better" than a cheaper electrolytic cap just because film caps last longer. Hey, if you're paying a few grand instead of a few hundred for an amp or CD player, you might expect to keep it awhile.
The flaws of DBT: Like everything of human origin, DBT is not perfect. But for minimizing or eliminating human bias in testing it is the best tool we have and is recognized as such in every scientific field. To paraphrase Churchill, DBT may be a bad system, but all the others are so much worse. And the common criticisms of DBT's validity (the switching is too fast, the stress of testing fouls up the results, etc.) in audio are simply based on ignorance of a great body of work on human perception, psychology, and hearing (an experimental psychologist I am slightly acquainted with describes many audiophile beliefs as "audio tarot"). AFAIK, the validity of well-conducted DBT is simply a non-issue to any genuine scientist. As I've said before: if we rely on DBT in life-and-death scientific fields like medicine, why should it be inadequate for the relatively trivial question of whether two amps sound alike?
As to the question:
What sort of listening tests do designers of audio equipment (amplifiers, preamps, CD players) consider to be valid for their purposes. In other words, how do they confirm, if only to overcome their own scepticism about their own judgments, that their designs sound as good as they measure?
I believe the answer is: DBT.
