Hacker News new | ask | show | jobs
by ouid 1833 days ago
Bonferroni effects have, at least in the popular consciousnes, become much more front and center. I wonder how many people on the stre can define p-hacking. Most educated people know about the reproducibility crisis, and I would wager that around half of them understand the cause. That seems like it might create a better environment.
1 comments

> around half of them understand the cause

I’m in the half that doesn’t know, apparently.

I considered that with hard sciences, the cost, time, equipment, conditions, etc were limiting factors. I think this paper on subatomic particles changing into antimatter is interesting, it was observed in a unique facility, once, under some condition that can never repeat, etc. Just kinda have to take your word for it.

As to soft sciences… I really don’t know there. Give me a hint?

Bonferroni effects. In the parent article referred to as researcher degrees of freedom, in pop culture referred to as p-hacking. In machine learning, it's referred to as overfitting. They are all the same phenomenon, of having more than one hypothesis, and then using statistical measures that assume that you only have one hypothesis to claim confidence. Journals only publishing positive results is an example of this phenomenon. Changing your hypothesis to match your data is another such example.