Velodyne Digital Drive Plus 18 (DD18+) Subwoofer Review

[pbc]

Brent's CEA test results are all over the map as can be seen with his DD15+ measurements that not only exceeded mine of the same sub, but even josh's measurements of the DD18+!

The same issue was noted via his measurements of the SVS pb12nsd.

Looks like Brent believes the AH testing methodology is incorrect and doesn't conform to CEA-2010 standards?

"With every CEA-2010 measurement I've done, I've tried my best to conform 100% with the standard, and I will continue to do so until if and when the standard is revised. Those who want to cite 2-meter measurements without adding +6 dB, or do voltage averaging instead of dB averaging, or measure without a correction curve, are the ones who IMHO are required to explain themselves."

Review of Def Tech 4000 Sub up on Sound&Vision.. - AVS Forum

 

[gene]

Looks like Brent believes the AH testing methodology is incorrect and doesn't conform to CEA-2010 standards?

"With every CEA-2010 measurement I've done, I've tried my best to conform 100% with the standard, and I will continue to do so until if and when the standard is revised. Those who want to cite 2-meter measurements without adding +6 dB, or do voltage averaging instead of dB averaging, or measure without a correction curve, are the ones who IMHO are required to explain themselves."

Review of Def Tech 4000 Sub up on Sound&Vision.. - AVS Forum

Brent is sorely mistaken.. None of his CEA #s are repeatable by the companies whose products he measures. He doesn't save the files to show his distortion plots so nobody can even look at his results. dB averaging is a useless and meaningless thing to do. I've already been in contact with CEA about eliminating it. We don't do any averaging b/c its meaningless.

There are three types of reviewers in this world:
1. those that blindly follow rules/standards
2. those that can think for themselves and recognize when a rule/standard is incorrect and find a better, more meaningful methodology to use
3. those that measure nothing and simply write on feeling

I'd like to think myself and my staff fit in the #2 category hence why we have our Room Size Rating Protocol instead of averaging in Voltage or dB which is meaningless. To Brent's credit at least he is not #3.

To my knowledge, Brent doesn't allow peer review of his work. He doesn't confirm his results with the manufacturer whose product he tests. This is NOT how we conduct our reviews and analysis. Believe what he says if you wish, but look at the caliber of reviews we are doing compared to his and make your own decision.

 

[Ricci]

I stand behind all of the measurements and data that I have done as being as accurate as possible, to the best of my ability and current understanding, and repeatable within close tolerance, under similar conditions with proper set-up and calibration of equipment. If the data is not accurate, or making sense within the realm of physics it will eventually become obvious and get questioned publicly. That is a certainty. That's about all I care to say on the subject of whether our measurements are accurate.

Happy Friday everyone!

 

[Ed Mullen]

I stand behind all of the measurements and data that I have done as being as accurate as possible, to the best of my ability and current understanding, and repeatable within close tolerance, under similar conditions with proper set-up and calibration of equipment. If the data is not accurate, or making sense within the realm of physics it will eventually become obvious and get questioned publicly. That is a certainty. That's about all I care to say on the subject of whether our measurements are accurate.

Our own internal PB13U data set is an extremely good match for Josh's data, as he indicated in the AH review.

Our data sets for each sub are reproduced at least twice within a 0.5 dB or better tolerance. All of our data is saved (including the raw notebook and packed experiment files).

"If the data is not accurate, or making sense within the realm of physics it will eventually become obvious and get questioned publicly."

This is really the key statement. If the data doesn't make sense in the physical or electrical realm, then there is a problem somewhere in the test rig. Buggy software, inaccurate correction factors, incorrect SPL calibration, a bad mic, a faulty cable, a change in the test environment - something.

We've logged hundreds and hundreds of hours of ground plane work, and we know exactly what our subs are capable of and what data sets they should generate. So when something looks screwy or doesn't make sense - we stop, troubleshoot, and find/fix the problem.

 

[gene]

Reagarding adding +6dB comment

Those who want to cite 2-meter measurements without adding +6 dB

I would like to address Brent's comment about adding +6dB:

The standard DOES NOT say you need to measure a sub at 1 meter.

Here is exactly what it says:

Place the DUT on the floor oriented so that its major radiating element faces towards the microphone,
which also rests on the floor (see Figure 2). Measurement distance shall be at least 1 m (3.3 ft.).

1 meter is a minimum and its usually better to measure at 2 meter for systems that have multiple drivers/ports to properly integrate their response.

We report our #s 2 meter with RMS approximation. That is because it is closest to actual 1m full-space anechoic output and makes more sense intuitively. It is also more in line with real world output seen in use than some "peak" 1 meter half-space number which is more about marketing.

If Brent/others are reporting at 1 meter, it should be clear in their documentation (IE. 1 meter peak). 2 meter RMS (how we report) can directly be converted to 1 meter peak by adding +9dB to each measurement. Even when we do this to the same subs we measured that Brent measured, his #s still come out much higher and have more variability at different frequencies.

We don't measure products to produce inflated #s for marketing. We measure for accuracy and we measure consistently so anyone, including the manufacturers can reproduce it. We also fully disclose all of our data which nobody else is currently doing when publishing subwoofer reviews.

 

[pbc]

Believe what he says if you wish, but look at the caliber of reviews we are doing compared to his and make your own decision.

I don't, I trust Josh's reviews as I know he is fully aware of what he's doing, and given others (such as Ed at SVS and I believe Brian at Rythmik) have come out to confirm their findings are within +/- 2 db's or better of his. Besides, it doesn't take a rocket scientist to look at even just the DD-15 review (and compare it to even Ilkka's old charts on various sealed boxes) to see that something is off on the SPL levels.

I just found the comment humorous, in particular "I do exactly what CEA tells me whereas others don't and it is they who need to prove themselves" so thought I'd point out his comment to you guys as I was guessing it was directed at you (I can't imagine who else it would be aimed at??).

Besides, these forums would be so benign without some controversy here and there.


I would be curious to see these "correction curves" that permit someone to do CEA-2010 testing indoors. I.e., per Brent's assertion that "If the measurements are done properly, with a correction curve (this process is outlined in the CEA-2010 document and explained in greater depth in the article I linked to), it does not matter whether they were done indoors or outdoors". I can't wrap my head around how that would work as I'd guess any correction values required would need to be extremely room dependant.

Subject for another thread I guess.

 

[pbc]

I would like to address Brent's comment about adding +6dB:

That comment had me chuckling as well, I would think that if someone mentions 1M vs 2M (and peak vs RMS) that, again, it doesn't take a math major to add or subtract 6 or 9dbs. Assuming the author of the test actually makes it known which methodology was used.

I just find with some reviews one never knows and is left guessing, and those who want to present these measurements in this manner are the ones who IMHO are required to explain themselves.

 

[cjwhitehouse]

"If the data is not accurate, or making sense within the realm of physics it will eventually become obvious and get questioned publicly."

This is really the key statement. If the data doesn't make sense in the physical or electrical realm, then there is a problem somewhere in the test rig. Buggy software, inaccurate correction factors, incorrect SPL calibration, a bad mic, a faulty cable, a change in the test environment - something.

We've logged hundreds and hundreds of hours of ground plane work, and we know exactly what our subs are capable of and what data sets they should generate. So when something looks screwy or doesn't make sense - we stop, troubleshoot, and find/fix the problem.

There is also the possibility of the unknown factor where you are not measuring what you think you are. If the subwoofer volume setting and test rig drive signal levels are not the same in both cases, it is at least plausible that a test rig with a high drive level and low subwoofer volume could drive the subwoofer into clipping more easily and therefore register a lower CEA-2010 limited max SPL figure to one where a lower drive is used and subwoofer gain set correspondingly higher. Both could still be technically correct by the standard and yield completely different numbers.

 

[gene]

There is also the possibility of the unknown factor where you are not measuring what you think you are. If the subwoofer volume setting and test rig drive signal levels are not the same in both cases, it is at least plausible that a test rig with a high drive level and low subwoofer volume could drive the subwoofer into clipping more easily and therefore register a lower CEA-2010 limited max SPL figure to one where a lower drive is used and subwoofer gain set correspondingly higher. Both could still be technically correct by the standard and yield completely different numbers.

If the person didn't know what he/she was doing I suppose its possible. An experienced reviewer knows NOT to clip their test rig.

 

[cjwhitehouse]

If the person didn't know what he/she was doing I suppose its possible. An experienced reviewer knows NOT to clip their test rig.

I'm not talking about clipping the test rig. I'm talking about clipping the subwoofer input stages.

 

[gene]

I don't, I trust Josh's reviews as I know he is fully aware of what he's doing, and given others (such as Ed at SVS and I believe Brian at Rythmik) have come out to confirm their findings are within +/- 2 db's or better of his. Besides, it doesn't take a rocket scientist to look at even just the DD-15 review (and compare it to even Ilkka's old charts on various sealed boxes) to see that something is off on the SPL levels.

Yep and actually our results are within 1dB of SVS for all subs we've measured for them. Few people know CEA testing as well or better than Ed Mullen which is why I consult with him regularly.

I just found the comment humorous, in particular "I do exactly what CEA tells me whereas others don't and it is they who need to prove themselves" so thought I'd point out his comment to you guys as I was guessing it was directed at you (I can't imagine who else it would be aimed at??).

CEA doesn't tell anyone to do anything. It's a guideline. It's up to the reviewer to come up with a valid and accurate test plan accordingly.

Besides, these forums would be so benign without some controversy here and there.

True b/c we don't have enough of that with Politics

I would be curious to see these "correction curves" that permit someone to do CEA-2010 testing indoors. I.e., per Brent's assertion that "If the measurements are done properly, with a correction curve (this process is outlined in the CEA-2010 document and explained in greater depth in the article I linked to), it does not matter whether they were done indoors or outdoors". I can't wrap my head around how that would work as I'd guess any correction values required would need to be extremely room dependant.

Subject for another thread I guess.

You cannot do CEA measurements indoors with the same degree of accuracy as outdoor tests regardless what section 4.3 of the CEA 2010 standard suggests as a procedure to follow. For an indoor correction curve to even remotely be correct, you would have to apply a unique one for each sub being tested. I wasn't aware Brent was measuring indoors but that may explain why his tests have so much variability and are often overly optimistic. His DD15+ results were WAY HIGHER than my DD15+ results and HIGHER than Josh's DD18+ results.

 

[gene]

I'm not talking about clipping the test rig. I'm talking about clipping the subwoofer input stages.

Again this should be obvious by increased distortion at all frequencies at even low SPL sweeps and the reviewer should be able to determine it and lower the drive level accordingly.

 

[pbc]

He confirms testing is done outdoors but he applies a correction curve as he is within a specified number of feet from a boundary (his house). Was just curious about these correction curves for indoor and outdoor testing and what exactly they were.

 

[MinusTheBear]

Audioholics IS THE STANDARD for measurement protocol these days. It isn't even close! For Brent to think he is even on the same playing field when it comes to measuring gear to what AH staff is doing is laughable.

Even if you look at S&V reviews for loudspeakers they are nowhere as in-depth as AH.

 

[pbc]

It is possible that Brent was referring to other reviewers as he didn't specifically mention AH, who else is attempting cea2010 these days anyhow?

 

[gene]

It is possible that Brent was referring to other reviewers as he didn't specifically mention AH, who else is attempting cea2010 these days anyhow?

Nobody to my knowledge. He is likely referring to us. I tried emailing him awhile back to discuss his measurement results and he did not want an open dialogue. It's too bad really as it would have been nice to have multiple review sites posting consistent results to allow consumers to compare apples to apples in case on publication hasn't reviewed a particular product another one has.

Also having multiple sources validate a each other prevents manufacturers from refuting test results if they aren't to their liking.

Oh well

 

[gene]

Update: CEA 2010a

As I said earlier, I've been in contact with the CEA 2010 working group.

Here is an email I just got from them today:

We have fixed our method for calculating average SPL per your earlier suggestion. Thanks for bringing that to our attention.

Our Audio Systems Committee discussed your proposed room size rating protocol late last year. While there seemed to be a fair amount of agreement that it is an appropriate method, there were some comments that it's not the only method. The group decided to go ahead with adoption of CEA-2010-A without incorporating your protocol so as not to delay adoption with the understanding that further changes can be made should consensus be reached on a room size rating protocol.

Somebody better tell Brent that the new CEA2010A standard now averages SPL in pascals instead of dB's

Regardless, averaging in dB's or Pascals is still not at all representative of anything meaningful especially if there are large variances in output at different frequencies. This is why a Room Size rating protocol like we developed is better or at least something that weighs the response.

excerpt from our Room Size Rating Protocol

It is mathematically incorrect to average logarithmic based numbers (such as decibels) as it will bias the outcome to the lowest number in the data set. The correct way to average dB's is to first convert them to Pascals (a linear, not logarithmic, measure of pressure) to properly average the data before converting back to dB's. There is a problem averaging like this, however, since our ears don't hear loudness equally for different frequencies. Averaging in Pascals will bias the average to the highest measurement in the data set. Properly averaging in Pascals (Pa) will make a sub with just one good SPL # still have a good average score. Averaging in dB’s (instead of Pascals) will make a sub with just one bad SPL # look bad.

Averaging across a bandwidth using discrete 1/3 octave wide data sets is just not a good idea any way you slice it. The speaker which is flat is going to achieve approximately the same score as one which is mistuned. If you tune too high, you get a bump followed by a quick rolloff. So if you average over too wide a frequency range, this information is obscured. 1/3 octave measurement is already a crude approximation for a sub. If you average that 1/3rd octave data over an even wider range, it just makes it harder to determine the quality of the product or measured data.

 

[Irvrobinson]

Wow...People really get bent out of shape over subwoofers.

Yeah, I was thinking about this phenomenon. Subwoofers do seem different. Maybe it's because subwoofer sound can be so understood and measured compared to full-range speakers. Subs are all omnidirectional, eliminating that factor. And for home theater at least it's a lot about sheer loudness. Main speakers are seldom like that. And look at all of the parameters you measure, including distortion. I think if main speaker sound was so easily (ha!) characterized people would get just as charged up about one of those reviews.

I also think subs are so... masculine, if I may be sexist for a moment. They're all about size and power. Look at that driver! How about that magnet structure! Bigger is better. Subs = testosterone-induced comparison checks.

 

[pbc]

I also think subs are so... masculine, if I may be sexist for a moment. They're all about size and power. Look at that driver! How about that magnet structure! Bigger is better. Subs = testosterone-induced comparison checks.

My family room is 1700 cubes and I have two dual opposed 15" sealed subs powered by a QSC RMX-5050 and the SVS PB-13 Ultra in sealed alignment providing me with sub 150hz. Slight over-kill. Am I compensating for something??

 

[pbc]

Nobody to my knowledge. He is likely referring to us. I tried emailing him awhile back to discuss his measurement results and he did not want an open dialogue. It's too bad really as it would have been nice to have multiple review sites posting consistent results to allow consumers to compare apples to apples in case on publication hasn't reviewed a particular product another one has.

Also having multiple sources validate a each other prevents manufacturers from refuting test results if they aren't to their liking.

Oh well

This is what is so unfortunate. Even though AHs/Josh's results seem to correspond almost perfectly with SVS and others while Brent's do not (well, except apparently the VTF-15 on its 2nd or 3rd try IIRC), if I was Brent I'd want to better understand the reasoning why. I.e., I'm sure everyone could learn from each other on how they do these measurements to make the process that much more reliable and better for all readers.

If I was a professional reviewer, I know I'd be keenly interested in ensuring my reviews were comparable to my peers, and if they were well off the onus would be on me to figure out why. The DD-15 just sticks out like a sore thumb in my opinion as the CEA numbers just don't seem realistically possible for any 15" driver in a sealed box that size with 1250 watts applied to it, let alone compared to your own DD-15 and DD-18 tests.

Instead now we have comments floating around of "CEA testing can result in drastically different measurements from one person to the next" to justify the differences.

Even though, heck, ignoring Ed's comments above, you could take Josh's PB13 measurements and compare them to Ilkka's from back in 2007 or 2008 within a couple db's across the board (save IIRC below 20hz), most of which is fully explainable by the fact that the PB13 Josh reviewed had the new amp which added 1 to 2db's by virtue of it's additional 250 watts or so. Two different subs, several years later!