Hacker News new | ask | show | jobs
by gamedori 2502 days ago
The previous thread was very critical of this article's statistics.

Reposting my comment:

When I see a headline like this my instinct is to jump to "overfitting". In a study like this they use P=0.05 as their cut off for significance. One in 20 such studies would be expected to give meaningless results. But they had the choice of cohort (all kids or boys and girls separately), so 1 in 10. They measured urine concentrations of fluorine at three time points during pregnancy, which makes for 7 different potential combinations of input variables, for a total of 14 different potential analyses, where 1 in 20 is expected to be significant. Of course, they didn't preregister, so who knows whether their analysis changed on the fly..

1 comments

Those multiple comparisons are quite obvious from the text, so a rigorous reviewer may have asked them to do a Bonferroni correction. Not sure if this was the case though.
You're right, and one would hope that happened. But not knowing the field or journal norms, all I could do was search the text for "Bonferroni" and "regist". In my lab if additional analysis was requested by the reviewer and we did it, we would have added a sentence to the manuscript.

But not having published in this field or journal, I have no idea what their norms are.