Certainly not every speaker is for everyone.
Some people prefer "more accurate" speakers, and some people prefer "less accurate" speakers.
Infinity P162/P362 are known to be "accurate" speakers, as confirmed by Soundstage/NRC and Stereophile.
NHT, PSB, Paradigm, Energy also fall in the "accurate" category.
Martin Logan speakers, as well as many other speakers like Polk, Klipsch, may fall into the "less accurate" category.
Of course, this is just generalization and does not apply to every case.
A lot of people think DefTech is no "accurate". But if you look at the DefTech Mythos STS (Soundstage/NRC) and SM450 (The Audio Critic), they are quite accurate based on their frequency responses.
A lot of the Klipsch speakers I've seen are not accurate either, but some are accurate.