Yes and no. Loudspeakers typically put out 2nd, 3rd, and 4th order harmonics. Note some of the waterfall plots taken on loudseakers. Then note audibilty of upper order distortions like 5th order distortion on up as opposed to the audibilty of lower order distortion. Once again, it's just not that simple, but yes I would like to see some of the measurements you have described. Who knows, it might embarass the loudspeaker industry enough to do something about it. ( O.K., I'm dreaming here)
In addition, I stongly suspect that many people are simply used to hearing certain kinds of distortion and "learned" to accept this as normal.
d.b.
P.S. Bongiorno did note that the musical scale does deviate from the "perfect" electronic scale, and he is correct. In the treble region the musical scale is "bent" so to speak, and I think this is one of the reasons that upper order distortion is considered harsh and aggravating. It's almost (?) a form of dissonance.