Hacker News new | ask | show | jobs
by bcdarwin 1985 days ago
Exactly - a p-hack is a form of misreporting, so isn't easy to detect at the level of individual studies.

However, p-hacking isn't the only threat to replicability: in this case the authors have reported a large set of tests, so we can ask why they didn't control family-wise error via Bonferroni or friends (in which case the reported statistical significance almost certainly disappears). Also, I suspect if you fit Bayesian models (either separately or a hierarchical model) using reasonably narrow priors based on what we know about human sensitivity to magnetic fields, how much other senses are affected by sex differences, hunger, etc., and not starting each test assuming a complete state of ignorance of the world, then the data would be compatible with no effect.