A/B Testing: Need Advice.

[Patrick_Wolf]

Is level matching required to do proper A/B tests?

I have the 4 pairs of contenders in my sig. I have one pair connected to my A terminals while another is connected to the B terminals and I can switch between them easily, but I can't level match them. So far I've been using my ears and adjusting the volume. I thought I was doing good until I tried to A/B the 2030P & TSBL. Now it just seems like I could be way off this whole time.

I'm thinking I can use my receiver's test tones (pink noise) and an SPL meter to read the decibels of one pair, then match it to the other by adjusting the volume. Then, when I switch between them I adjust the volume however many clicks it takes to match up. Would this work?

 

[Robof83]

You could use a spl meter to level match. Level matching can be difficult with speakers because of the wide swings in frequency response. I would suggest using a 1khz test tone to level match. Set a value for each speaker so you know in advance what to turn it to. Just before you switch you should pause and adjust the volume.
Also I would suggest doing the evaluations with some kind of blind fold or cloth over speaker on, and have someone else to switching if thats at all possible. Thats the only way to no for sure that you are getting the best sounding speaker to your ears.

 

[lsiberian]

Is level matching required to do proper A/B tests?

I have the 4 pairs of contenders in my sig. I have one pair connected to my A terminals while another is connected to the B terminals and I can switch between them easily, but I can't level match them. So far I've been using my ears and adjusting the volume. I thought I was doing good until I tried to A/B the 2030P & TSBL. Now it just seems like I could be way off this whole time.

I'm thinking I can use my receiver's test tones (pink noise) and an SPL meter to read the decibels of one pair, then match it to the other by adjusting the volume. Then, when I switch between them I adjust the volume however many clicks it takes to match up. Would this work?

This is going in your house not a science lab. IOTW the thing needs to look your style as well as sound good.

There is more to speakers than just the sound. The speaker can't be ugly as pie after all.

 

[Loren42]

This is going in your house not a science lab. ....

I agree. If you are comparing to see which ones you like best, just play each set with the same sets of music and see which pleases you most.

It is probably more important that you get a representative sample of the music you intend to play on a daily basis and give each a test run.

 

[Patrick_Wolf]

This is going in your house not a science lab. IOTW the thing needs to look your style as well as sound good.

There is more to speakers than just the sound. The speaker can't be ugly as pie after all.

No, for me it is only about the sound. Maybe you think I'm being to anal about it, or you don't understand what I'm asking.

I'm trying to level match because of the variances in sensitivity between each pair. Maybe this isn't possible to get it precisely the same? But I read they used pink noise in the 2007 & 2009 bookshelf shootouts to level match each pair. That's kinda what I'm after. I just want the comparisons to be fair.

 

[Patrick_Wolf]

I agree. If you are comparing to see which ones you like best, just play each set with the same sets of music and see which pleases you most.

It is probably more important that you get a representative sample of the music you intend to play on a daily basis and give each a test run.

I did try that. I'm still trying that. It's not working to well. Because to me they all sound good. Different, but still good. And no I'm not going to just pick the one I think looks the best cause I really don't care.

 

[fredk]

It sounds like your volume knob has discrete clicks for each volume level. If that is the case, your method should work fine. It would be kind of challenging if the knob is a continuous pot.

And no I'm not going to just pick the one I think looks the best cause I really don't care.

I didn't much care about looks either. I stepped down a finish to step up a level in speakers because they sounded noticably better in a properly level matched a/b comparison.

Good luck with your comparisons. Let us know how it works out.

 

[Robof83]

Aye, I'd be very interested to hear your results when you finish.

 

[Patrick_Wolf]

It sounds like your volume knob has discrete clicks for each volume level. If that is the case, your method should work fine. It would be kind of challenging if the knob is a continuous pot.

I didn't much care about looks either. I stepped down a finish to step up a level in speakers because they sounded noticably better in a properly level matched a/b comparison.

Good luck with your comparisons. Let us know how it works out.

I can use the remote to change volume in increments of 1. That's my clicks, the knob on the receiver is continuous though.

My comment about speaker looks wasn't directed at you, was just a general comment.

I thought I had the TSBL and SBS-01 eliminated from my A/Bing, but I have doubts about that now. I will keep at it. I think I just need more time with them.

 

[snowmanstu1110]

Very interested in this post. I just recently got in my Beta 20's, and have TSC P6's and TSBL's coming this afternoon.

But I was in the same boat, when I was comparing the Onkyo's HTIB speakers to the Beta's. I had them hooked up to the A/B inputs on receiver don't have level match calibration available. So it's down to finding equal levels by ear (or spl) and then pausing the playlist, turning down the volume, and switching. It's a pain but it works.

And just in case anybody is curious, the difference between the Beta's and Onkyo HT-6100 HTIB speakers is laughable not like that's a surprise.

 

[Pyrrho]

Is level matching required to do proper A/B tests?

Yes, absolutely. Human hearing is such that a slight difference in volume is perceived as being tonally different. As the volume decreases, subjectively, the bass and treble appear to diminish faster than the midrange. Thus, two otherwise identical speakers, if playing at very slightly different volumes, will appear to produce different amounts of bass and treble. This, by the way, is what those "Loudness compensation" switches were all about on older gear, so that when listening at a soft volume, one could boost the bass and treble in a way to try to compensate for this peculiarity of human hearing.

This also explains why all gear, when doing A/B tests, needs to be level matched, or the test has no validity. Since virtually no one does this, it means that the opinions of people regarding the relative sound quality of different pieces of equipment is almost always worthless.

I have the 4 pairs of contenders in my sig. I have one pair connected to my A terminals while another is connected to the B terminals and I can switch between them easily, but I can't level match them. So far I've been using my ears and adjusting the volume. I thought I was doing good until I tried to A/B the 2030P & TSBL. Now it just seems like I could be way off this whole time.

I'm thinking I can use my receiver's test tones (pink noise) and an SPL meter to read the decibels of one pair, then match it to the other by adjusting the volume. Then, when I switch between them I adjust the volume however many clicks it takes to match up. Would this work?

It will probably work okay. However, I am not an expert on these matters (though I do know a lot more than the average person about it, that does not make me an expert at all).

My guess, for what that is worth, is that you will find that one speaker is better in some ways, and another is better in other ways. I remember auditioning speakers many years ago, and being frustrated that one kind of drum sounded more real on one speaker, and another kind of drum sounded more real on another. One I liked better overall, but I did not like it better in every way.

 

[mtrycrafts]

...The speaker can't be ugly as pie after all.

They do make acoustically transparent material to drape over them

 

[mtrycrafts]

I did try that. I'm still trying that. It's not working to well. Because to me they all sound good. Different, but still good. And no I'm not going to just pick the one I think looks the best cause I really don't care.

It will be tough to level match speakers, unless you have Harman labs help as each have different sensitivity, etc. You just cannot do it by ear. Try the pink noise and spl meter and see how close they come.

 

[Patrick_Wolf]

An Update.

I haven't done anymore A/Bing as I don't have an SPL meter (yet).

I did what Loren42 suggested and picked a handful of mp3's of varying genres, a few scenes from my limited DVD collection, a bunch of apple traillers, and a couple of games to go through with each pair to see which I found the most pleasing. I went through each pair consecutively so the sound was fresh in my mind.

For each pair I adjusted the volume to a comfortablly loud level. Trying to keep it the same for each one. For movies I tended to adjust the volume for dialog.

I should note that I'm listensing nearfield, so room accoustics are of less importance... I think?

Well I was right initially, the SBS-01 & TSBL are out. I'm not sure I'm using these terms correctly but here I go.

These all sound good, but there can be only one so I have to nitpick.

The SBS-01 sounded a bit too laid back/reserved, and it's overall presence just seemed smaller than the others; probably due to it's size. And maybe lacking in dynamics a bit. I would agree with Tom's review in that they're average for music, but I wouldn't say they're outstanding for movies. Definitely good though.

The TSBL is hard to describe. It's bass is strong, but I think it tended to overshadow the mids/highs at times. Also I felt it may have been lacking in fine detail a bit. And voices seemed a little thin compared to the others. It's difficult to detect, but where I really noticed it was 300 on BD; the narrator's voice sounds noticeably different.

So it's down to the 2030P & CBM-170SE... again. After the past two days of listening I realized why I like them so much. Listening to them is just effortless. They're so smooth & neutral, but far from boring.

I know WmAx says the 2030P is slightly superior as it has less cabinent resonance & the CBM-170SE has an increased treble band (or something) that gives them a harsher sound. The 170SE can of course get twice as loud, but I'm nearfield so it's not a concern.

Thanks for the responses so far. I'm going to keep listening to the 2030 & 170. I'm not sure if I'll order the M3's yet to compare as I hate eating up these return shipping fees.

 

[Patrick_Wolf]

2nd update for anyone who cares.

As far as I can tell the 2030 & 170 sound virtually identical. I found it very difficult to detect even a slight difference. Even though I bought the 170's b-stock, I couldn't possibly justify the increase in price over the 2030's.

I tend to get obsessive, as such I ordered the M3's (shipping back the M2's using FedEx online label's said $21 total, but the charge on my credit card was only $16.20 total.. so fairly cheap anyway). I'll probably spend some more time with the 2030 & 170 while I wait for the M3's.

 

[fredk]

Excellent! It sounds like you are making good progress and have a good idea of the sound you like.

 

[Kitsum]

Last year, when it came down to select one of my four pairs and Ebay the others, i couldn't decide which one i liked best.
After weeks of A/Bing and listening, i decided to keep all of them. Now i just cycle between pairs every few weeks or months. It keeps my music fresh and my listening interest high. Change is good.

 

[Robof83]

Last year, when it came down to select one of my four pairs and Ebay the others, i couldn't decide which one i liked best.
After weeks of A/Bing and listening, i decided to keep all of them. Now i just cycle between pairs every few weeks or months. It keeps my music fresh and my listening interest high. Change is good.

Very interesting. Never heard that one before

 

[Lordoftherings]

Last year, when it came down to select one of my four pairs and Ebay the others, i couldn't decide which one i liked best.
After weeks of A/Bing and listening, i decided to keep all of them. Now i just cycle between pairs every few weeks or months. It keeps my music fresh and my listening interest high. Change is good.

I got several brand of speakers in my main system, and I also switched them once in a while,
just for a new fresh sound.
They are permanently positioned too. It looks very professional, just like those show rooms
where they test various speakers side by side.
I thing it's a great idea, and the way I positioned them, they don't interfer with each other.
Plus, Audyssey MultEQ XT does a good job at that too.
Change is indeed good.

Bob

 

[fredk]

I'm not nearly obsessed enough about this audio thing. I only have one set of speakers.