Hacker News new | ask | show | jobs
by JacobDotVI 1759 days ago
Research in general is the same, but shouldn’t _published_ research have already gone through a filter such that it represents the 10% that hasn’t failed?
1 comments

Only if you tie publication to reproduction. But you can't really do that, because to reproduce an experiment independently, the independent researchers need to have learned about your result and methodology, which = publication.

Without being able to independently verify results, a lot of journals rely on signals like your research & results being noteworthy, which is anticorrelated with them being true. If our understanding of the world is at all correct, you'd expect most experiments to give boring, status-quo results, and your first impulse when you get something noteworthy is to assume the experimenter made a mistake. (Interestingly, this is how things work at the undergrad level - if you measure the speed of light at 1 million meters/sec, your prof will assume you screwed up.)

Probably we need a separate verification step after publication, with researchers evaluated on what fraction of their published results survive verification, not just how many papers they publish. Publish all the noteworthy results you get, but assume that published research doesn't mean much until it's been independently reproduced.