[tomlock]

> Combined with industry-wide pressures to publish, the replication crisis was inevitable.

> The replication crisis, if nothing else, has shown that productivity is not intrinsically valuable.

I think this is important to focus on - the point of universities has become to produce profit, and to give people degrees that are profitable, and to appear to be able to do those things. This has very little to do with producing research with verifiable results. It's much more to do with getting students into the funnel by making people with tenure appear as productive as possible.

[nine_k]

AFAICT this is not about profit in a commercial sense, as in selling goods.

It's more like overfitting the target function of publishing impactful research. A bit of p-hacking, a bit of cutting corners in experimental setup, a sloppy null hypothesis check, and you honestly believe you see an effect! Everyone is happy: you, your adviser, lab's administration, the journal where you publish the paper.

But if you carefully check for everything, then find no effect, you kill an interesting hypothesis, your paper is hard to publish, "you are not making progress", and nobody is happy.

Crooked incentives, crooked results :(

[rleigh]

I think if you take it further, the incentives are ultimately financial: promotion, better salary, continued employment. The alternative is losing your funding and getting the boot.

[tomlock]

I agree that what you say is happening is happening, I think there was some great meta-analysis that showed that p-values were not following a distribution that was statistically possible - like on OkCupid where people that are over 5'10" round up to 6ft.

But I think the underlying reason for the push to publish things - and impactful things are easier to publish, is profit. More hireable grads, more tenured professors publishing papers.

[pryffwyd]

I have read that a study is very unlikely to replicate if p is juuust under 0.05, but very likely to replicate if just over 0.05. The first is a good sign of p hacking, while the second is a good sign of a real effect with a sample size that wasn't quite big enough.