As for the audibility of group delay in low frequencies, I have not heard of any rigorous scientific testing of delay perception below 500 Hz, and I have looked, through JAES archives, JASA archives, and IEEE archives (yes, they do publish research in audiology). The 20 ms thing is kind of a informal guideline within certain parts of the audio industry, and there is no research to back that up. Among the findings, it takes 10 ms to identify a 500 Hz tone correctly 90% of the time for healthy human hearing, and that is double the duration of the cycle.
Here is an informative graphic related to the subject:
"Pitch also depends on duration. A tone must last more than a minimum amount of time (~10-60ms, depending on frequency and intensity) in order to sound more than a 'click' and convey a clear sense of pitch (see the figure, below)."
What this says to me is that audiophiles have a far greater estimation of the acuity of their hearing than the reality.