If there are subtle difference, then one would need speakers of high enough calibre to resolve these subtleties plus an acoustically treated room to hear them, aspects that are missing from amplifier tests.
I've always thought that there were several kinds of differences:
1. Differences that are inaudible. (Got that out of the way.)
2. Differences that are audible, but innocuous. Like frequency response being down 2db at 18KHz. To some people it will be audible, but it probably only on test tones. Some differences will not lead a person to think the amp is inferior. A bit of second or third harmonic distortion may fall into this category. In fact, some people may like it.
3. Differences that are audible and overt. Like old-fashioned clipping.
4. Differences that are audible but not overt. I think these differences contribute to listener fatigue, but may not be easily perceivable at any given time. I'd put IM distortion in this category, as well as compression (like from input stage overload), or perhaps a noise floor that is really non-linear in its frequency distribution. I'm not so sure super speakers or super rooms are required to perceive these differences.
I think class 2 and 3 distortions mostly don't exist in well-designed solid-state amps. I think people like me who believe there are some audible differences between amps are talking about class 4 distortions in my list, and they annoy you over time. I think this could be why it is so difficult to do even a simple A/B test with amps and reveal repeatable preferences, yet some of us perceive them. Or on the other hand class 4 differences are really class 1 differences, and people like me are deluding themselves.