flaws in Christine Tham's articles

krabapple

krabapple

Banned
This has bugged me for awhile. Audioholics hosts a couple of articles by Christine Tham that come to conclusions not warranted by their methods. In particular there's one on Myths of Digital, the other on the relative dynamic range of LPs vs CDs. Both are (unintentionally I'ms ure) misleading to the reader because their conclusions derive from flawed methods. I respectfully suggest that they be substantially revised, or withdrawn.

For a discussion of the flaw in her Digital Myths report, see posts on hydrogenaudio starting from mine here:

http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=47827&view=findpost&p=428338


For flawed practice in the LP vs. CD article :

http://www.delback.co.uk/lp-cdr.htm#record_resolution

(EDIT: and if a moderator could insert the word 'in' into the subject line, I'd be grateful ;> )
 
Last edited by a moderator:
Perhaps you can direct the more knowledgable people on those forums to construct a new test sequence that is more real world? We're open to additional input.
 
Axon

Axon

Audiophyte
I'll bite. However, I will say that I have no professional experience with audio performance evaluation.

So here are the issues I see with the dynamic range comparison of LP to digital.

  • RMS calculations are used as a metric for equalized loudness, which I think is not correct. Signals must be passed through an equal-loudness filter of some sort, or at least A-weighting, to evaluate if they are equally "loud".
  • The algorithms used for calculating virtually all of the numbers are not broken down, which makes it difficult to impossible for another person to replicate the results. RMS window sizes are not broken down. FFT window sizes are not broken down. FFT window functions are not broken down.
  • In general, Mrs. Tham is not actually comparing the dynamic range of LP vs the digital formats, she is merely comparing the dynamic range of recordings of various albums that are released in different formats. As she notes, this introduces many unknowns (the quality of the records, different mastering choices, compression, etc) that compromise the strength of her results. Moreover, this grossly underestimates the actual, numeric dynamic range of CDs. Well, actually, it underestimates the dynamic range of all the formats.
  • Comparing peaks of time-domain waveforms is technically misleading and any results based on the comparisons, in my opinion, are not meaningful. They might be corrupted by: phase distortion in the RIAA filter, phase distortion in the reverse RIAA filter during laquer cutting. harmonic distortion on recording or playback, mistracking, and record damage. Again, there are just too many unknowns that can't be factored out.
  • The single-number RMS figures Mrs. Tham gives include the energy contributions from rumble, which, as she points out, would grossly favor digital formats in a numerical comparison. Since these recordings are mainly for technical analysis rather than listening, a high-order IIR highpass filter at 50hz or so on both the digital and LP recordings would eliminate all rumble and facilitate more even measurements.

To actually test the real performance of each medium, I think there's no choice but to use test records. One can be constructed easily enough for CD. For LP, the Hi-Fi News test record is servicable, although HFS75 or STR151 (or most CBS test records for that matter) would also work.

I'll admit that Mrs. Tham's test sequence as it currently stands is a lot more "real world" insofar as it uses real music material rather than artificial tones. But I would argue that, when entering a topic as technical as dynamic range in the first place, using music material to measure it is meaningless. You absolutely must use test material specifically designed to measure it.

What we're looking for here is a test tone, at a known calibrated physical amplitude, which can be compared against the average background noise of the recording on a frequency-dependent basis. This ratio forms a signal-to-noise ratio. If the tone is believe to be at or near the maximum amplitude supported by the medium, then it also becomes the dynamic range. (The SNR of vinyl is much less than its dynamic range, due to headroom restrictions. The dynamic range of a digital format is usually the same as its SNR.)

Next, simply using a single RMS number for a signal is not sufficient for evaluating dynamic range, because of the varying effects of harmonic distortion compared to the RMS energy at any single frequency. What I believe is correct is to compare the peak RMS amplitude at a tone's fundamental against the average RMS amplitude of a range of noise frequencies.

Finally, using an excessively long FFT window length will reduce the accuracy of the LP measurements, not improve it. Because of speed variations in LP playback, if you use a FFT with a high enough frequency resolution, the test tone will move back and forth on the FFT, reducing the tonal energy at any one frequency, while leaving the noise energy undisturbed. This could reduce the measured dynamic range drastically. To get optimal results you need to use a window length that is insensitive to 1-2% of speed variation. For the 300hz tones on HFNRR the window length should be absoutely below 14000 points, and 2048 points is probably close to optimal.

To improve the resolution and consistency of the FFT results you'd need to average all the FFTs together. I'm not sure if CoolEdit will do this.

So, to put it all together, here's a test sequence:

  1. Grab the Hi-Fi News and Record Review test record, or, failing that, HFS75 or STR151, or, failing that, STR100. (Change the directions below as appropriate.)
  2. Create a test tone at the highest resolution possible (192/24, or 96/24) for a 30-second, 300hz tone at 0dbFS. Downsample it to 16/44 at the highest quality possible - noise shaped dither, max quality resampling algorithms, etc. Create a test CD with a 30-second track, at 300hz, at 0dbFS maximum amplitude.
  3. Record the outer band, 300hz, +15db track on side 2 of HFNRR. Trim it to contain only the full-signal tone and no silence. Record it at 44.1khz - not 96khz - because your ADC will invariably run at lower noise at 44.1 than at 96 (unless you know otherwise). Use 24-bit or 32-bit recording.
  4. Record the test CD track. Trim it. Also record at 44.1/24.
  5. Run FFTs on every 2048 points of each track. Use a windowing function designed for high dynamic range, but also doesn't have a flat spectral leakage response asymptotically. This means: good choices are Nuttall, Blackman, Hann; bad choices are Rectangular, Hamming, Hann, Gauss, Bartlett, Blackman-Nuttall, and Flat Top.
  6. Square the FFTs.
  7. Average the FFTs together.
  8. Normalize the FFTs so that the 300hz tone has a maximum at 0db. Read out the amplitude at all other frequencies to obtain your frequency-dependent dynamic range readings (but obviously ignore the measurements at harmonics of 300hz). To convert the LP measurements to SNR instead of dynamic range, subtract 15db.
 
Last edited:
S

Sonicus

Enthusiast
krabapple said:
This has bugged me for awhile. Audioholics hosts a couple of articles by Christine Tham that come to conclusions not warranted by their methods. In particular there's one on Myths of Digital, the other on the relative dynamic range of LPs vs CDs. Both are (unintentionally I'ms ure) misleading to the reader because their conclusions derive from flawed methods. I respectfully suggest that they be substantially revised, or withdrawn.

For a discussion of the flaw in her Digital Myths report, see posts on hydrogenaudio starting from mine here:

http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=47827&view=findpost&p=428338


For flawed practice in the LP vs. CD article :

http://www.delback.co.uk/lp-cdr.htm#record_resolution

(EDIT: and if a moderator could insert the word 'in' into the subject line, I'd be grateful ;> )
Hmmm, in the LP "cleanup" articale - what was the point of here writing the article without even spending some quality time with the three big name programs that people might ask about? Surely, this would have much more useful to us readers.

Also, I wonder if she considered that the "de-crackle" exercise she got from the "youngblood" thread will result in the same outputas if she had just done the same function without inverting the "leftover crackle"?

DOn't know much about her articles, generally, but she does seem to woffle on :)
 
S

Steve1000

Audioholic
Very interesting. Welcome to audioholics. I hope you'll stick around, and bring some of the other people from hydrogen with you. The spirit is definitely right here, but the level of expertise is not always as high as you find at hydrogen.:)

Axon said:
So here are the issues I see with the dynamic range comparison of LP to digital....
 
krabapple

krabapple

Banned
Oh jeez, and I just noticed you've updated her Digital Myths article, but it still comes complete with the old 'digital can't do square waves or sawtooths' false alarm! And have the brass to call the article 'Dispelling Myths of Digital' (last modified April 2007').

http://www.audioholics.com/education/audio-formats-technology/exploring-digital-audio-myths-and-reality-part-1

(hint:: no one hears a saw tooth, or a square wave, just as an oscilliscope presents it; that's because human hearing is band-limited, as are all recording and playback media, including analog ones. Tham skirted close to the right answer, then shied away by invoking amplitude distortion at 19.99kHz and musical signals that are 'very much like' sawtooths).

Don't you guys review this stuff? If you don't see the problems with it, might I suggest you offer to pay some real authority like Richard Pierce, Jim Johnston, Bob Katz, Bruno Putzys, Nika Aldrich, Dan Lavry, or others from the Pro Audio list (pgm.com) to write such articles, instead of this *amateur*, whose erroneous methods and conclusions get tediously propagated through the internet audio forums? :mad:

Here's Pierce on the 'square wave' canard:

http://groups.google.com/group/rec.audio.high-end/msg/4d71ed5066b7f71c?dmode=source
 
Last edited:
C

charonme

Audiophyte
Dynamic Comparison of LPs vs CDs - Part 4

I just came accross this article: audioholics.com/education/audio-formats-technology/dynamic-comparison-of-lps-vs-cds-part-4 (Dynamic Comparison of LPs vs CDs - Part 4) because I was searching for some LP vs CD comparison articles and I was also puzzled by the methodology. Additionally to other things mentioned in this thread, Chris Tham is not really comparing dynamics of LPs and CDs, but just digial "remasters" or rips and CDs, because the LPs are first ripped to a digital format! Even if the recordings came from the same source with the same processing and even if there was a significant advantage of the LP, the difference should not be measurable by digitizing the LP and comparing with the digital release!

I would suggest using a device which could measure the various parameters of signal dynamics in the analog domain without digitization (I have no idea whether such a device even can exist) and then comparing the LP signal with a setup where there would be a A/D - D/A element inserted between the LP amp and the speaker amp.

It would be good if the opposite approach would be possible, too: make an LP using a digital master and then rip the vinyl and compare the digital original with the digital rip, but I personally believe that the first test would show that the LP and the LP-AD-DA would be sufficient to show that CD can represent the LP signal sufficiently for listening.
 
F

fmw

Audioholic Ninja
All of this is funny in many ways. Recording engineers hotly debate whether it makes sense to record anything with more than a 16 bit depth. After all, the medium we use to listen to the recording is 16 bits. In the pop world they compress and limit the mix to the point that 12 bits would cover it just fine.

So the the debate is about an audible difference between a 16 bit recording or something like a 24 bit recording dithered to 16 bits. I think most of them record at 24 bits simply because it doesn't cost them anything to do it. It is one of those "just-in-case" sort of things.

I'm not a computer scientist but I do have some expertise in objective listening tests. I can tell you that nobody has ever been able to hear a difference between a 16 bit recording and the same recording made at a 24 bit depth then dithered to 16 bits in a bias controlled test. You can print all the graphs and O-scope traces you like, but it just doesn't matter. Whatever differences they might point out aren't audible. It is an academic argument, not a practical one.

We get the same thing in consumer audio with issues like biwiring. People can argue all day long that biwiring differs electrically from "mono" wiring but, in the final analysis, that electrical difference doesn't produce an audible difference in bias controlled listening tests. So it is akin to arguing about the number of angels that will fit on the head of pin. Purely academic.

I didn't read the original article but, if the author said it makes no sense to record a vinyl record at more than a 16 bit depth just to dither it back to 16 bits to burn a CD, then the author is right if the issue is audibility. If the issue is graphs or o-scope traces, then go chase some angels from the pinhead. It doesn't really matter.
 
WmAx

WmAx

Audioholic Samurai
I didn't read the original article but, if the author said it makes no sense to record a vinyl record at more than a 16 bit depth just to dither it back to 16 bits to burn a CD, then the author is right if the issue is audibility. If the issue is graphs or o-scope traces, then go chase some angels from the pinhead. It doesn't really matter.
The OP's response to the original article was pointing out that the article is highly flawed, so much in fact, that it has little in the way of valid information, and could actually be construed as mis-information and should not be hosted by this site, or at very least, be highly revised.

-Chris
 
newsletter

  • RBHsound.com
  • BlueJeansCable.com
  • SVS Sound Subwoofers
  • Experience the Martin Logan Montis
Top