This may prove to be an unpopular view here in AH, but that's exactly why it's in the SV section. It has to do with choosing what equipment to buy - listening vs. measures (what is the correct ratio?).
My point first; I'd even give less credit to listening than most people here do, but would never opt against, of course.
Now the explanation; personal experience is highly overrated in today's world and I believe it needs no advocates. The reason it is overrated is because it fuels the impulsive purchase. As Liev Schreiber nicely put it in Repo Men: we need them to buy, not to think! Which I personally really don't like.
On your way to finding your speakers (or any product), the market prefers those who try and miss, hitting their head on the wall several times ending up in spending 10k for a 2k product rather than those who arrive at their goal spending the exact amount or a "mere" 4k for a 2k product.
Simply put, the market loves you uninformed.
That's why whenever someone tells you to just listen, no matter how well intended it may be, I think he only helps the ones I don't care for all that much.
Few notes on senses. There is a certain way the brain processes the sensory input. To keep it as short as possible, the brain discriminates among the input and also adapts to some of them.
Nose. Have you ever met a lady that keeps reapplying the perfume she really likes and have you noticed that there's a wide spread opinion that "those perfumes that smell good last shorter"? It is important for human to be able to detect a dangerous scent even in a heavy scented environment. This is why after a short while the brain will disregard all the unimportant input. This is way you can sense new odours even in a cesspool.
Sadly, the very fact that the perfume smells nice to you will make the brain stop reacting to it sooner - as it is not dangerous and the brain needs to concentrate on other input so it disregards it (but for the love of... ladies, don't reapply!!). And for you, gentlemen, this is a huge part of your 'upgrade bug'. Sometimes you seek to be thrilled all over again because you got accustomed (brain adapted) to what you own. Also, as it was once said here in AH, you're inclined to go for the 'different' sounding speaker in the shop because it stands out and this you see as a promise of that thrill. Then it ends up on Audiogon among ZuAudio.
Eyes. Do you wear spectacles or know someone that does? Did he/she ever tell you how the doctor told them that they need to adjust dioptre, but that the new and better match will actually bother them in the beginning? The same thing. The brain sets your spectacles to be the new null, it adapts to it. It is telling your consciousness (let's just leave it, please), that everything is fine with your vision. It needs to pass a certain threshold (limen) in order to perceive it as a problem.
Have you ever experienced coming to your friends house and seeing that all of the reds on his TV are closer to orange (or that the black has a lot of green...), but he sees nothing wrong with it and can discern among orange and red. Same thing.
Same goes for ears. If the deviation in sound is of no vital importance, the brain will disregard the deviation. This is of grater importance for beginners. After being exposed to PC desktop speakers for a few years or some ancient farting membranes, you're much more inclined to reach for a manipulated speaker.
Knowing all this, I couldn't advise someone, with a clear conscious, to just go out there and listen.
I could say this; if you're very experienced and can afford it, sure, just listen, buy, sell, exchange... It's all fine.
If you're a newbie, listen among the well measuring products and start your adventure there. Don't go by your ears alone. Most of 'love at first sight' is short lived.
In today's market your 'Hit n' Miss' will probably be 1% to 99% respectively.