[_Codemonkeyism]

From the linked article:

"A score in the bottom 5% would typically mean there is a problem. But Shura pointed out that the diplomats were found to be “impaired” if they scored in the bottom 40% in any one of the tests. In their letter, the doctors said it was “inappropriate” to conclude that any of the patients were impaired. "

[hmahncke]

It’s unfortunately quite challenging to diagnose cognitive impairment based on neuropsychological assessment at a single point in time post-injury. That’s because we don’t know what the person’s level of cognitive function was before the injury. If a person had cognitive function 1 standard deviation above the mean pre-injury (equivalent to an IQ of 115, speaking loosely) and suffered a 1 standard deviation loss as a result of the injury, they would then test in the normal range - but still clearly have suffered damage.

The response articles calling for a 5th percentile measure to diagnose impairment are in a sense statistically appropriate - you’d like to see very significant impairment to diagnose it from a single measure - but not sensitive - they will miss a lot of true cases of mild impairment. For example, that threshold would miss virtually all cases of concussion, chemobrain, cognitive impairment in MS, etc.

[ScottBurson]

What these doctors are missing is the power of statistical correlation to separate signal from noise. Yes, a single individual testing below 40% means nothing, but 20 people (or however many it was) all from the same office testing below 40% is starting to be significant.

[_Codemonkeyism]

They've tested 6, don't know how they drew the sample.