|
|
|
|
|
by thatcat
1985 days ago
|
|
It's actually up to the paper to prove that it's not a p-hack. That's done by declaring a null hypothesis and the intended stats analysis to be done before doing the experiment and collecting the data, because otherwise you can just compare arbitrary things until you find that some of them are correlated in the set you have. There's no numerical post-hoc analysis that would be able to prove that. |
|
However, p-hacking isn't the only threat to replicability: in this case the authors have reported a large set of tests, so we can ask why they didn't control family-wise error via Bonferroni or friends (in which case the reported statistical significance almost certainly disappears). Also, I suspect if you fit Bayesian models (either separately or a hierarchical model) using reasonably narrow priors based on what we know about human sensitivity to magnetic fields, how much other senses are affected by sex differences, hunger, etc., and not starting each test assuming a complete state of ignorance of the world, then the data would be compatible with no effect.