Hacker News new | ask | show | jobs
by ip26 2340 days ago
Really? Suppose you just ran a twenty year, many million dollar field study, the hypothesis was not confirmed, you just throw everything out & that's that?
2 comments

No, you publish a negative result, which in many ways can be more or just as informative as a positive finding. See for instance https://www.nejm.org/doi/full/10.1056/NEJMsa1906848.

It's totally fair to do subgroup analysis, etc to try to better understand the data, but you have to keep in mind that you can't claim statistical significance on those without very careful control over researcher degrees of freedom and other related issues.

Yes because firstly, you didn't design the experiment for that reason, so you didn't consider what inadvertent biases you introduced for the "new" way of looking at the data. Plus on top of that, because of false positives, you're going to find a LOT of false positives by accident.

So every study "discovers" something new, but it's not because of biases and false positives.