Neumi BS5 Bookshelf Speaker Review and Analysis

[ErinH]

First off, here's the link to the review via my site. There is additional information there that I am not including here. I am just covering the highlights here. If you want more details look at the review page here: https://www.erinsaudiocorner.com/loudspeakers/neumi_bs5/


Intro
I stumbled on talk of the Neumi BS5 speaker recently in the context of a potential high-value speaker. Out of curiosity, I went to the product page on Amazon to check them out and liked what I saw. I then pulled up the Neumi’s BS5 manual here is the link where I saw placement recommendations, and some other bits of information which all gave me the impression the manufacturer cares about how the user listens to their product rather than the old “sink or swim” attitude low-cost products leave you with. Generally, when this information is laid out for the user it also implies the product is worthwhile. At least, that’s the impression I am left with in those cases.

At any rate, Amazon had them listed for $90/pair (at the time of purchase) and I figured they were worth buying to review and pass the information on to the audio community so you all could either avoid them or feel comfortable spending your hard earned money on. So, I did.

Ultimately, while these aren’t the best performing speakers I’ve tested or heard, I do believe these provide a good value to the budget-limited audiophile. And, with a few engineering alterations, could be made into an even better value. Read on for more detail.




Product Specs and Photos

Impedance Phase and Magnitude:
Impedance measurements are provided both at 0.10 volts RMS and 2.83 volts RMS. The low-level voltage version is standard because it ensures the speaker/driver is in linear operating range. The higher voltage is to see what happens when the output voltage is increased to the 2.83vRMS speaker sensitivity test.

Frequency Response:
The measurement below provides the frequency response at the reference measurement axis - also known as the 0-degree axis or “on axis” plane - in this measurement condition was situated in-between with the woofer and the tweeter per the product manual. While the manual does do a good job of directing the user how to set up the speakers, I emailed Neumi to ask about listening angle and the grille use. I wanted to make sure I used the speakers the way they were designed to be used. Below is our email exchange:

I purchased your BS5 bookshelf speakers and was wondering: Are these designed to be listened to on-axis (with the speaker aimed directly toward the listener) or at some angle off-axis? I assume the former. Are these designed to be listened to with the grilles on or off? I assume off, as most speakers perform worse with the grilles on. Thank you.

Hi Erin, Thank you for your inquiry! The BS5 are designed to be listened to pointed straight forwards. If you like to have a slightly brighter response, you can point the speakers more towards the center position. We also tuned the BS5 without a grill. The grill was made afterwards to minimize its effect on the speaker output. It is fairly transparent but does change the response slightly.

If the speakers are to be aimed facing forward, that would be approximately 30-degrees off-axis in my room. I can toe them in or out if you recommend using a different positioning angle than this.

Hi Erin, Thanks for the additional information. I would start out pointing straight, then try it with 10-15 degree toe-in and see how that sounds to you, more than that, the toe-in would be pretty extreme and is not recommended.

So, per Neumi’s direction I listened to the speakers both on-axis (0°) and off-axis (≤30°) horizontally. I found the best angle to be directly on-axis. Otherwise, the treble was too subdued. When it came time to measure the speaker, I verified that 0° gave the most linear response and conducted the rest of my analysis with the reference axis being at 0° horizontally and between the mid/tweeter vertically.

Also, per Neumi’s direction, the grille was off for these measurements. I do have comparison data of the grille on vs off in my Miscellaneous section below.

CEA-2034 (aka: Spinorama):

What we can learn from this data is that this speaker has significant directivity problems thanks to the deep nulls at ~800Hz and ~1600Hz. You can see it in the above spectrogram and globe plots as well as in all the measurements in the above graphic. The crossover is stated as 2.1kHz by Neumi and the nearfield data backs this up. Therefore, in this region you can see the directivity mismatch. Looking at 1kHz you see a rising DI until approximately 2.5kHz where the Early Reflections DI dips back down again. This is a sign the transition from mid to tweeter is occurring as the woofer is beginning to beam (radiate more forward than omnidirectional) and the tweeter is taking over, omnidirectional until approximately 6.5kHz (calculated based on dome size of 1 inch). The DI flattens out a bit through here but as the tweeter begins to radiate more directionally the DI increases again above ~7kHz. The tweeter rolls off sharply above 16kHz, causing directivity to increase further. What does this all mean to you? Well, mismatches in what is coming directly at you, on-axis, vs what is reflected around you can cause issues in stage and tonality cues.

Below is a breakout of the typical room’s Early Reflections contributors (floor bounce, ceiling, rear wall, front wall and side wall reflections). From this you can determine how much absorption you need and where to place it to help remedy strong dips from the reflection(s). Notice the strong dips again at 800Hz and 1600Hz.

Total Harmonic Distortion (THD) and Compression:

Using the 93dB measurement tells you the measured low-frequency distortion at about 80Hz is near 3% THD and 6% at 40Hz. Will you hear that? Pure distortion is more subjective and depends not just on the listener but also no the program material.

I typically use distortion to tell me where mechanical failures are because the distortion I hear is typically either a rattle, buzz, plop from a woofer extending too far, or something along those lines. The bass is usually the problem. But in this speaker the midrange exhibits distortion at higher output levels and was also audible in my listening (primarily with male vocals).

The compression effects shown in the image below are a visual way of seeing just what happens as the volume is increased. This one is straight-forward. Take the legend’s SPL value and add or subtract the data from the graphic. This tells you if you’re losing or gaining output (yes, you can gain output from compression; as un-intuitive as that seems). Mostly, the compression results in a loss due to temperature increase in the voice coil of the drive unit. Let’s look at a specific example. Take the 90dB at 4 meters target listening volume provided above. Again, you need 93dB’s (7.62vRMS) data. At that volume, the highest amount of compression measured is about 1dB at 40Hz and about 0.25dB at 50Hz, decreasing until about 200Hz. At some points the speaker suffered >2dB compression at 40Hz with 14vRMS. Overall, the compression results tell you what common sense would tell you: don’t try to use this speaker in place of a subwoofer at anything other than lower volumes. Otherwise, at louder listening volumes you lose over 1dB of output. And it is audibly present as a very grainy and “limited” sound; there are no dynamics at this output and that’s exactly what I heard in my listening tests when I pushed the speaker to uncomfortable levels.

Extra Measurements:
These are just some extra sets of measurements I completed. Some, I didn’t process through my MATLAB scripts so they’re kind of raw. But I know some would like to see them so here you go.


On-Axis Response comparison between Speaker A and Speaker B.

This result shows a very good consistency between units other than the response below 30Hz. I’m impressed!

Grille on vs Grille off at 0° and 45°.

The grille on case results in an increase in comb filtering (higher amplitude peaks/dips). Leave the grille off.

Accelerometer Cabinet Resonance Test.

I used an accelerometer. I attached the accelerometer to the side (top, middle, bottom), back and front of the cabinet and fed a 2.83vRMS signal through the speaker.

I have overlaid all these results against the on-axis response as well. Note, however, the accelerometer is NOT calibrated for level and therefore is only intended to show relative changes. What you look for in this data is large peaks that would indicate a resonance. I’m seeing a lot of mountains and molehills, but I don’t think this data is conclusive enough to do anything other than make a mountain out of a molehill. Others may disagree.

Since the accelerometer is not calibrated for voltage in the above graphic, I chose instead to view the results via a Sonograph, shown below. This shows a long decay for 800Hz - 900Hz. This seems to line up with a resonance of the ports, shown in the next section.

Nearfield measurements.

Mic placed about 0.50 inches from each drive unit and port. While I tried to make these as accurate in SPL as I could, I cannot guarantee the relative levels are absolutely correct so I caution you to use this data as a guide but not representative of actual levels (measuring in the nearfield makes this hard as a couple millimeters’ difference between measurements can alter the SPL level). Got it? Good.

There are a few noteworthy things here:

• Port resonance is very, very strong and clearly contributes to the on-axis response dips at ~800Hz and ~1600Hz.
• The area between 300Hz to 700Hz (just before the 800Hz dip) is elevated slightly. This area also lines up with the increased THD levels I discussed earlier. This could be coincidence. But I believe they are related. Maybe the port is having more of an effect in this region than it needs to?
• Woofer break-up contributes to a few on-axis resonances we see. Particularly, 4.5kHz.
• There are other things going on here but I don’t have the time to reverse engineer this speaker. Not that I could.

Step-Response.

One not zoomed and one zoomed.

Plugging the port (making the speaker sealed).

To test whether the ports were, indeed, the culprit of the deep nulls I took my socks off and plugged the ports. Don’t worry, I had only been wearing the socks for 3 days. Sure enough, plugging the ports filled in the nulls. But it also decreased the low frequency output by about 2dB below 300Hz.

Objective Evaluation:
Much of what I am about to say I have already touched on under the data. But to recap:

Impedance:

• Minimum load of about 4.6 Ohms. But mostly > 6 Ohms. Check your receiver or amplifier’s spec to make sure it can drive a 6 Ohm load without issue.
• Wiggles around 200Hz and 280Hz indicate resonance which also shows up in frequency response.

Frequency Response/Spectrograms/Globes/Spinorama:

• I measured an average of 84.2dB @ 2.83v/1m.
• I measured a ±3dB response from 64Hz - 20kHz but only if you ignore the dip in response around 800Hz. Neumi claims a ±3dB window of 50Hz - 20kHz (typical in-room). Buy a subwoofer if you want to listen loud and low.
• Numerous resonances; most caused by the port. Woofer breakup shows up in a few places as well.
• Directivity shifts caused by inadequate crossover order and resonances from the ports and woofer.

Distortion/Compression:

• High distortion at 40Hz but understandable given woofer size.
• High levels of compression at high output below 100Hz.
• Elevated midrange distortion (audible at higher volumes).
• These are both audible effects when listening full-range as I did.
• Don’t expect much bass below 80Hz out of these speakers. Buy a subwoofer for that.

If more time/money were spent on taming the resonances and break-up modes I think this speaker could be markedly improved. But, for $90, you kind of expect these things. Namely because higher order crossovers are not cheap and take up real-estate.



Subjective Evaluation:

Subjective Analysis Setup:

• The speaker was aimed on-axis with the tweeter at ear level.
• I used Room EQ Wizard (REW) and my calibrated MiniDSP UMIK-1 to get the volume on my AVR relative to what the actual measured SPL was in the MLP (~11 feet from the speakers). I varied it between 85-90dB, occasionally going up to the mid 90’s to see what the output capability was. In a poll I found most listen to music in this range. Realistically, 90dB is loud for long-term listening volume and I find most overestimate their listening volume until an SPL microphone is used to determine the actual level.
• All speakers are provided a relatively high level of Pseudo Pink-Noise for a day or two - with breaks in between - in order to calm any “break-in” concerns.
• I demoed these speakers without a crossover and without EQ.

I listened to these speakers and made my subjective notes before I started measuring objectively. I did not want my knowledge of the measurements to influence my subjective opinion. This is important because I want to try to correlate the objective data with what I hear in my listening space in order to determine the validity of the measurement process. I try to do a few listening sessions over a couple days so I can give my ears a break and come back “fresh”. I also want to be as transparent to you as I can be so below are my subjective evaluations made before I began any measurements.

Here’s the rundown of my subjective notes (in quotes) and where it fits with objective:

• Overall, I found the max SPL I could drive the speakers to was around 90-92dB at my listening position, depending on the music. That’s loud. But once I got past this point the compression was very audible and all the dynamics went away. This was most evident on the opening bass notes of Lauryn Hill’s song. It was very evident that I had reached the “brick wall” output here, even though the woofers weren’t mechanically falling apart like I would have expected.
• In my listening tests the main thing that stood out to me was the high-frequency balance being off. In some cases it sounded about 1-3dB too low. In a few cases I heard some ‘sizzle’ on instruments that I do not believe are correct (I didn’t make the album; I can’t know for sure). The data tends to agree with that in relation to the rest of the spectrum. There are some hot spots here and there discussed previously.
• I felt room ambiance was lacking in some recordings. For example, I noted this in “Higher Love”.
• I made a few notes about resonance in lower vocals and questioned if I could “hear cabinet ringing”. I noticed this primarily in “Everybody Wants to Rule the World” and “Tell Yer Mama”.
• I noted midrange distortion at ~ 90dB (at 11 feet) in both Jim Croce’s and John Mayer’s tracks. I wasn’t sure what this was when I listened the first time, but the data clearly shows an increased level of distortion smack in the middle of the midrange. I went back through a final round of listening after I saw the data and on the “He Mele No Lilo” track, at the end, I could hear distortion in the singer’s voice. It seems I noticed this distortion in male vocals.
• I noted some things that I hadn’t heard or wasn’t used to hearing with other speakers. For example, Chaka Khan’s voice as background singer in Steve Winwood’s “Higher Love” was more present. I don’t know what to attribute this to… is it a distortion in the midrange? Is it the breakup from the midwoofer at higher frequencies? Is it a relative thing; the dips at ~800Hz and ~1600Hz causing other areas to be more noticeable than a more flat speaker?
• The bass was punchy; the harmonics of kickdrums and synth sounded good. But there was no weight to those because the speaker just doesn’t play that well below about 100Hz.
• The stage width was a weird one. In some songs it didn’t seem wider than 10° outside the speakers (so, about 40° total) where with other songs it was wider than this. Each recording is different, and you do want a stereo system that expands and contracts proportionately with the music. But I believe the variance in this case is more attributable to the directivity changes caused by the crossover and resonances.

I also turned the speakers to be about 10 to 30° off-axis to see if I could get rid of the harsh treble. That didn’t help much at all and when you view the data you can see the off-axis response has low directivity around 4kHz (meaning, the sound is more omnidirectional at this frequency) which indicates the bright 4kHz region would be noticeable through a wider region of angles. I believe this explains the “biting” noise I was hearing as well.

I didn't have a chance to run Dirac Live so I can't speak to what the sound would be post room correction.




Bottom Line
This speaker doesn’t measure perfectly. And, for the most part, I was able to match areas of concern between my subjective listening session and my measurements. Though, I didn’t have any significant gripes about the sound. The one main dislike for me was the reduced treble compared to the midrange. The bass is pretty well blended to the midrange despite the moderate bump in response around 100Hz. There is not much output below this at higher levels, but I can forgive the shortcomings in the bass department because the BS5 isn’t trying to pretend that it can play like a subwoofer. I have seen other 5-inch woofers with higher linear excursion than what these woofers are seemingly capable of but just one of those drive-units alone costs more than this pair of speakers. The midrange distortion is an issue if you’re going to listen at high levels; for me being at 90dB at 11 feet (which is about 93dB at 8 feet per this awesome calculator). These aren’t reference level speakers. But I think anyone buying them understands the implicit output limitations. Under 90dB at 11 feet, the sound is more balanced and undistorted.

Personally, I think these speakers would be better suited as desktop/computer speakers sealed (stuff the ports) and against a wall. The wall would give you a +6dB increase on the lower end to help make up for the plugged ports but plugging the ports would get rid of the nasty resonances that plague this speaker. I would not place these in a corner in a small room, though. Doing so creates a combing effect you do not want. Alternatively, you can use these as small satellite speakers for a budget-minded home theater. However, if you want ultimate hi-fidelity at reference levels on a shoestring budget then these speakers are not it. The frequency response deviations and distortion keep it from that goal. But, when used within reasonable limits, this is a “fun” little speaker that is enjoyable and a great entry into the hi-fi realm at $90/pair. I hate using the “but it’s cheap” argument but, really, this is a $90 pair of bookshelf speakers. More than that, though, there’s no marketing language by Neumi to suggest they are the best speakers ever. Nothing that overstates their capabilities that I have seen. I think Neumi had a target in mind with this price and performance and I believe they hit it.

I’m going to plug my Amazon affiliate link one last time just in case you want to buy these. I know, I know… I’m a sellout.
https://amzn.to/2Abda9w



The End
If you like what you see here and want to help me keep it going, you can donate via the PayPal Contribute button at the bottom of each page. Testing and reporting the data and analysis takes me approximately 8-10 hours each. It’s definitely a labor of love. That said, there’s no fame or fortune in this and all my test speakers are typically purchased and paid for by myself with help from contributions or purchases made through my affiliate links (which is negligible). Your donations help me pay for new test items, shipping costs, hardware to build and test, etc. Even a few dollars is more helpful than nothing. If you don’t mind chipping in a few bucks now and again it would truly be appreciated.

Here's a direct link to contribute.
https://www.erinsaudiocorner.com/contribute/
Again, any bit is really appreciated. I would love to be able to fund a remote controlled turntable for my measuring. As it is, I walk about 2 miles (literally) between my computer and the DUT to spin it about 150 or so times (ground plane measurement + free-field measurement) at a distance of 40+ feet one-way which adds up.

 

[shadyJ]

Another great review. I hadn't even heard of these speakers before but they seem decent for the cost. I have to wonder how the Dayton MK402X or MK442 would fare against them, or the Micca stuff.

 

[afterlife2]

I saw these on amazon about a week ago and posted about it here on a thread. Glad to see that these are a worthy purchase. Gonna go read your review now. Thanks in advance.

 

[ryanosaur]

That vertical dispersion though... its a b!tch. Looks like you are better off having those speakers placed slightly below ear level, but just barely.
Everything else looks nice on first appearance.
Maybe except that XO, but hey... I'm still building my first. I'll get back soon when that is done.

 

[ErinH]

Another great review. I hadn't even heard of these speakers before but they seem decent for the cost. I have to wonder how the Dayton MK402X or MK442 would fare against them, or the Micca stuff.

Someone else asked about comparing them to the Micca MB42X and I wouldn't mind testing those out because I'm tapped out on my site funds. Until I can get enough contributions to cover the cost of other stuff I am on testing hold. Unless someone has a pair to loan me and can cover shipping.

Amir did test the Micca RB42 and it performed poorly:

Micca RB42 Bookshelf Speaker Review

This is a review and detailed measurements of the Micca RB42 bookshelf/desktop computer speaker. I purchased my sample from Amazon and it cost US $149 including Prime shipping. This speaker also seems to be the favorite of youtube bloggers, helped partly by generous seeding of such...

www.audiosciencereview.com

 

[ErinH]

That vertical dispersion though... its a b!tch.

Yea. The port resonance isn't doing them any favors, either. It's causing the listening window to deviate as low as 500Hz when it really shouldn't start until closer to 2kHz.

This was a tough speaker. If it were $150I'd say to save your money and wait for better or find something better used. But at $90/pair I think it's a good first step in to the realm of audiophile speakers.

 

[ryanosaur]

An interesting question becomes where the cutoff lies for good quality audio speaker vs low budget, and what is possible at that price point.

I think a real good example is the Dayton BR-1 kit and the Dennis Murphy Affordable Accuracy Mod. This whole thing started with modded AJ Pioneers IIRC:

Affordable Accuracy Monitor Review

This is a detailed measurements of a modified Pioneer SP-BS22-LR speaker by @Dennis Murphy called Affordable Accuracy Monitor (AAM). He was kind enough to send me a sample. I let him explain the mods but basically the tweeter was replace with a higher quality one and a new modified crossover was...

www.audiosciencereview.com

...and then evolved.
I own the AA version using the BR-1 with the original Dayton Tweet. It is currently dissected waiting for me to finish the next mod he did which is replacing the Dayton Tweeter completely with the Morel MDT-32S. The original Dayton woof performs quite admirably, and as you trace the evolution, the AA+ (Morel) version definitely crosses over into audiophile quality performance.

The New Affordable Accuracy Plus Monitor

 

[shadyJ]

Someone else asked about comparing them to the Micca MB42X and I wouldn't mind testing those out because I'm tapped out on my site funds. Until I can get enough contributions to cover the cost of other stuff I am on testing hold. Unless someone has a pair to loan me and can cover shipping.

Amir did test the Micca RB42 and it performed poorly:

Micca RB42 Bookshelf Speaker Review

This is a review and detailed measurements of the Micca RB42 bookshelf/desktop computer speaker. I purchased my sample from Amazon and it cost US $149 including Prime shipping. This speaker also seems to be the favorite of youtube bloggers, helped partly by generous seeding of such...

www.audiosciencereview.com

Some manufacturers are pretty eager for exposure and would be happy to send you out stuff, especially if you aren't charging anything for the review. It doesn't cost anything to ask the manufacturer if they want a review.

 

[ErinH]

Some manufacturers are pretty eager for exposure and would be happy to send you out stuff, especially if you aren't charging anything for the review. It doesn't cost anything to ask the manufacturer if they want a review.

That is true. I usually try to avoid having to go that route so it doesn't appear as bias or skew my review. Especially when I feel they won't like what I have to say. lol

But, it's always surprising what you can get when you simply explain your goal and ask nicely. I've been surprised how willing people in this community are to help each other in this regard.

 

[ErinH]

Heads up...

After I tested these speakers I sent Neumi the link. I have since developed a nice back and forth with the folks at Neumi (they're really quite nice). I asked if they might be willing to provide a pair of the BS5 speakers for giveaway purposes via my Facebook group page and they obliged. Yea!

So, if any of you are on Facebook and interested in entering the giveaway, join my group at this link:
https://www.facebook.com/groups/607627396679113/

I'll probably post the details there in the next few days. I'll just ask folks to 'like' a post so I can tally the names and do an online generator for the draw. I do this a lot with other things.

** I get not everyone is on Facebook. I know some hate it. But that's the easiest way for me to set this up. **

 

[slipperybidness]

I like that their pics on Amazon describe the driver integration as "harmonious"

 

[KEW]

This is a minor criticism, but I still think it is worth making the comment.

If I don't have much going on, I will enjoy reading a speaker review. However, if I am feeling a bit of time crunch, I approach a review like this with the question "Do I want to read this review?"

I believe the two biggest factors for this decision are:
1) What does it cost?
2) How good is it?
There is no easy set of rules for what will interest me (for example, if a very expensive item fails miserably, I am curious what got screwed up).

You provide this info in the last two paragraphs of your intro. If that is the structure you will continue to use, I now know where to look.:

At any rate, Amazon had them listed for $90/pair (at the time of purchase) and I figured they were worth buying to review and pass the information on to the audio community so you all could either avoid them or feel comfortable spending your hard earned money on. So, I did.

Ultimately, while these aren’t the best performing speakers I’ve tested or heard, I do believe these provide a good value to the budget-limited audiophile. And, with a few engineering alterations, could be made into an even better value. Read on for more detail.

So, I know where to find the info I am after. However, I believe it would be beneficial to make this info more accessible by using a format similar to what Audioholics uses to put these two nuggets of info front and center to the presentation.

Dayton Audio MK442T Transmission Line Tower Speaker Review

Dayton Audio's new MK442T tower speakers at $228/pair feature a transmission line enclosure for bass. Read our review to see how they managed to produce a quality speaker at a ridiculously low price!

www.audioholics.com

Last, I would also like to see any unusual aspect of a speaker pointed out in this "quick info" section, stuff like "this is an active speaker", "this speaker incorporates a transmission line system for bass", "this speaker uses an internal woofer with a passive reflex driver in front", etc ... because that is also information that will help me quickly decide if and how carefully I want to read a review!

Whether you adopt these thoughts or not, it is wonderful to have another source of comprehensive and accurate measurements of speakers!

I am a fan and made a donation! (I point this out in the hopes of getting others to do likewise)!

 

[KEW]

I like that their pics on Amazon describe the driver integration as "harmonious"

Yeah, I once had some speakers with inharmonious driver integration and they were, like, the worst!
I learned my lesson and now I only buy speakers that have harmonious driver integration!

 

[ErinH]

I appreciate the feedback. And no offense taken at all.

I do agree that if I am going to continue with providing text reviews in full that I need to have a BLUF. I did that in the Neumi review. I flat out told people the Jamo S807 was terrible in the previous review. I did not do that in the Buchardt review. But I blame that on me getting ahead of myself. My goal was (and still is) to use my website as a 'repository' of sorts to host the data but use YouTube videos as the channel (pun not intended) for review. I will discuss the product in the video and use my site in screen-share mode to discuss the data. That saves me from having to spend hours and hours typing a review, explaining all the data every time I review a product. It would be nice to just plop the data on the site and use the video format to explain it. I still plan to do that. I just let myself get sucked in to typing these long reviews because I fear someone viewing it out of context without the videos. I *plan* to start the video process this weekend, though.




As for using a numbers or stars system...

I tend not to discuss value. That's subjective. I may have a much harder time spending an extra $500 to get a bit better performance than another. I've been through this with many when I test drivers. Scanspeak Illuminator drivers are an easy example. They are $250+/each for certain models. Do they perform better than $50 drive units? Yea. Do they perform $200 better?... well ... that's up to the person who is shopping. They may not need to pay extra money for an improvement in an area they won't benefit from. I can't really assign a monetary value on something for anyone other than myself. In rare cases, like these bookshelf speakers, I come across an item that is pretty much at the low end of the cost spectrum and as such all I need to know is: does it perform well enough to purchase? If so, then it has value. If not, it doesn't. I can (and do) either tell you if I think a product is a good performer and I tell you the various aspects to its performance both subjectively and objectively. I would absolutely love to own a pair of Revel F328Be. I cannot afford them. I am certain the performance is incredible. Would it ever be considered a "value"... certainly not to me because they cost as much as my vehicle. But it is up to the person reading/watching to make the decision if it is worth their money. I know if I had enough money I would own a completely different system than I do now. But it's not worth me having to walk 40 miles one-way to work.

Not only that, but in some ways you can paint yourself in to a corner using numbers. I've seen it done by other reviewers in the past. At some point things will change (technology gets better, you get a better understanding or better measurement equipment) and that high value may not be such a high value. Something else comes along better than that and it is the "new" 4 out of 5 and the "old", were it to be tested against the current standards, would be moved down to a 2 out of 5. It's a sliding scale and there is little objectivity to these kind of values. I try to avoid subjectivity to the best of my ability because of things like this.

Using a performance metric is more reasonable. But that's also up to some interpretation. There is a prediction calculation to determine how much a person might prefer a certain speaker based on Sean Olive's work. It's used heavily at ASR now. And it is solely based on objective data. Unless I were to incorporate that calculation in to my work then there will always be some degree of subjectivity in relating how various metrics equate to a value. And you can literally determine performance/$ from it. But I do not use it myself because I do not know all the details at this point and I would rather not project misinformation in lieu of being able to have a number.

I believe what I do now is fair and reasonable. Provide the data. Discuss the pros and cons. Say whether I think it is a good speaker. And let the viewer decide if it's worth their money to buy it.


This may seem as if I am putting too much thought in to. To which I would reply, maybe others don't put enough thought in to it. That's not a dig. Just an observation.

 

[shadyJ]

One thing I was going to suggest at first but decided against it, was that you have a 'short version' of your reviews and then a long version like what you do now. But it looked to me like you are more interested in having the most comprehensive review available and that a short version isn't necessary. But if you don't already know, having that many graphs is probably a turn-off to a lot of readers. Most people have trouble following these graphs and will tune out as they get overwhelmed. Only speaker nuts like us will be that deeply interested. That might sound strange coming from me who posts lots of graphs in my reviews, but I might actually get more views by eliminating some of the graphs, however, getting rid of too many graphs would sacrifice the quality of the review in my opinion. One thing we at Audioholics have noticed is that too many graphs lead to less views.

I think you probably have given these issues some thought and that your conclusions have led you to make the reviews that you have. I am glad for that, and I think your reviews are terrific, however, I have heard and measured a fair number of speakers, and I have more experience than most people in correlating measurements to sound character, so your reviews are more illuminating to me than say the average person. So just a thought, in order to gain more traffic to your site, maybe you could make a short, easily digestible page for layman and neophytes, and then have the comprehensive page for nerds like us.

 

[ErinH]

Oh, trust me, I know that data overload can cause people to lose interest. I get it. But I would rather have too much data and be able to thoroughly review a product than have too little for the sake of getting a couple extra folks' views. My site isn't revenue based. This is just something I enjoy doing. Sharing data, helping others learn and getting an excuse to play with some cool toys!

The data ultimately will supplement the video reviews. The videos will be focused on reviewing the data and, ultimately, provide more people with insight in to what it all means.

 

[KEW]

Not only that, but in some ways you can paint yourself in to a corner using numbers. I've seen it done by other reviewers in the past. At some point things will change (technology gets better, you get a better understanding or better measurement equipment) and that high value may not be such a high value. Something else comes along better than that and it is the "new" 4 out of 5 and the "old", were it to be tested against the current standards, would be moved down to a 2 out of 5. It's a sliding scale and there is little objectivity to these kind of values. I try to avoid subjectivity to the best of my ability because of things like this.

Using a performance metric is more reasonable. But that's also up to some interpretation. There is a prediction calculation to determine how much a person might prefer a certain speaker based on Sean Olive's work. It's used heavily at ASR now. And it is solely based on objective data. Unless I were to incorporate that calculation in to my work then there will always be some degree of subjectivity in relating how various metrics equate to a value. And you can literally determine performance/$ from it. But I do not use it myself because I do not know all the details at this point and I would rather not project misinformation in lieu of being able to have a number.

I agree that the rating systems are problematic. Here on Audioholics it is further compounded by the fact that there are many different reviewers. It is probably unreasonable to expect an individual to maintain consistency with himself (or maybe I should say a lack of contradiction), but if we have a rating by Andrew Gash 12 years ago, it is ridiculous to really expect James review this week to be dead-nuts consistent with it!
Perhaps a simple thumbs up or thumbs down (and maybe a "meh" for in between). I like Amir's panthers, they give it a bit of whimsy, but he still gets the occasional "How can this ..., if this ..." criticism/question!
I don't think there will ever be a perfect answer to this.

Here's a thought, if you forum software allows you to conduct polls (as AH does), you could have a poll at the top of the review and let readers rate it. This makes it obvious that the rating is not based on any rigorous analysis. I think it would probably result in a decent system as long as you don't have fan-boys blindly up-rating everything by a specific maker or down-rating everything by a different maker! Like I say, "a thought" I'd have to let it percolate for a few days and even then like to do it on a trial basis to see how it would play out!

Edit: Playing with the poll idea - perhaps you could limit the product ratings to a sub-category of members of your forum (like people with a certain number of "likes" of seniority, or donors, etc)?

 

[ErinH]

Neumi BS5 Bookshelf Speaker video review is now up!