Hacker News new | ask | show | jobs
by jerrytsai 3211 days ago
Definitely. The main problem is that in the current system no one is being rewarded for good science, but for showing something interesting, bolstered by a declaration of (statistical) significance. The incentives are not aligned with societal objectives.

Good science requires a tension between hypothesis generation and skepticism. Perhaps if we rewarded the _debunking_ of findings as much as we do the discovery of findings, things would change.
1 comments
adrianratnapala 3211 days ago
Why doesn't this happen already.

The funding bodies etc, who want "quantitive" measures of research look at publications. Why would we expect debunking papers be published if they are debunking something interesting?

link
adrianN 3210 days ago
Once you have debunked something interesting I suppose you don't have trouble getting in published. But you have a hard time writing a grant application that reads something like "I want to replicate study X, no new results are expected."
link
adrianratnapala 3210 days ago
I think this hits the nail on the head.

That said, it also seems like low hanging fruit. At least in some fields, replication should be a lot cheaper than doing things in the first place -- because a lot of the cost in pissing about trying to find something that even seems to work.

For funding bodies to explicitly support replication studies, even if each gets only 20% of the amount the original studies get should be a winner at reasonably low cost.

link
nonbel 3210 days ago
No, because they want to present the conclusions of each project as a "fact" that we now know for certain, as in a constant march of progress. They feel it reflects poorly on them if they published/funded something incorrect, so they come up with any excuse to not publish any critique or do a retraction. Eg, Andrew Gelman has some pretty good stories about this:

"In the review process they did not disagree with my points at all, but they refused to publish the correction on the grounds that they only publish the very best submissions to ASR." http://andrewgelman.com/2016/02/22/its-too-hard-to-publish-c...

link