Hacker News new | ask | show | jobs
by staticautomatic 2358 days ago
I bet a lot would change if the largest funding institutions publicly prioritized funding scientists who have a history of publishing reproducible work.
3 comments

Sure - for some definition of "reproducible". Any metric you come up with that's easily measurable will be gamed into oblivion. Anything meaningful - like looking at dedicated reproduction studies, etc. - requires a complete change in how academia functions, because right now publishing a paper that just aims to reproduce (or fail to) a previous work is effectively impossible in most fields. Not to mention that given the incentive structure you'd be pretty crazy to even try.
It may well be some kind of law that every positive incentive creates at least one perverse incentive, but that doesn't mean positive incentives don't work or aren't worth trying.
I tend to argue that 10%-20% of a grant should go to reproducibility. This does two things:

1. It would help fund / ensure funding to several labs in the space.

2. It would help ensure reproducibility of results

Yes, it can be gamed, but generally it should be easier to reproduce results so 10%-20% of the funding to follow directions should be okay. Of course, this could lead to one group just constantly doctoring results to show something is not reproducible. In which case, a third lab would need to get some funding to check it out.

First they'd have to fund the reproducibility studies.
Well they shouldn’t have any trouble funding the first one...
They have plenty of trouble funding much of anything.