[kansface]

Via HN yesterday [1]- an editor of _Anaesthesia_ did a meta study of the papers he handled that conducted RCTs. He had data from 150 of them and concluded:

> ...26% of the papers had problems that were so widespread that the trial was impossible to trust, he judged — either because the authors were incompetent, or because they had faked the data.

This is not a one off.

[1] https://www.nature.com/articles/d41586-023-02299-w

[tptacek]

I didn't say it was a one-off. But 150 papers is, to a first approximation, a one-off of all the science done in a given year. We produce millions of journal articles every year.

[esyir]

There's something to be said about a defense of this that doesn't account for random sampling.

Assuming that they did a proper sample of said papers, that implies that for whatever domain they sampled, 26% is likely a decent estimate of actual issues. Increasing the scale doesn't make a proportional estimate any better.

[Auracle]

Maybe we shouldn't. What's the point of all of that data if a good portion of it can't be trusted?

[tptacek]

Here we're talking about a proportion significantly less than 1%.