[thaumasiotes]

> I don't know how to fix it other than several governments and their education ministries making a joined effort to have scientific papers where each result has to be thoroughly reviewed by multiple other labs. It's expensive, but I don't see other ways.

This is just aggravating the problem. Science is mostly fraud because it's mostly done on behalf of a funder who doesn't want it. It's easier to write a paper describing the results you'd like to get than to actually get those results. And the funding agency is indifferent between those two things. So mostly you just get the papers.

[Ashkee]

Feels like the incentives are backward and it pushes people to publish fast instead of verify. You could try HifiveStar to track and surface trustworthy feedback signals across sources, it helped me sift noise from solid input. The result is fewer wild claims and more time spent on work that holds up.

[epolanski]

How would that be aggravating the problem?

The current system has essentially no requirement of reproducibility.

Having a paper that only allows reproducible experiments (where there's funding for random labs to reproduce results) may be difficult, or an utopia, or whatever, but not aggravating for sure.

[thaumasiotes]

It's aggravating the problem because you're proposing to put that indifferent funder in charge of ensuring quality. There are two problems here:

(1) This is not a recipe for actually getting any quality.

(2) By virtue of providing the funding, they already are in charge. They're not going to get better results by wishing harder. But they can easily waste more money than they already do.

Your mental model seems to be that the government received a mandate to cause research to happen, and they did that as faithfully as they could, with the only problem being that we forgot to specify that we didn't want fake research. So if we change the mandate to "cause non-fake research", the kind of research we get will change.

But that makes no sense. "Non-fake" was always a requirement. It was an unenforced requirement because it didn't matter to anyone, but you aren't proposing to change that.

[fuzzfactor]

>requirement of reproducibility.

Check my other comments.

Reproducibility can be a working requirement before publication when the progress is expected to be serious.

At ASTM the publishing company is non-profit and more non-academic industries pay for the (not cheap) publications every year. The employees are well-paid journalists and efficient bureaucrats specializing in continuous quality improvement themselves, highly skilled at organizing the scientists. The scientists are all volunteers.

>indifferent funder in charge of ensuring quality.

Nice not to have. Publication requires complete consensus of the volunteer scientists, and the institution is crafted to progress toward valid consensus.

It's all about quality from day zero.

In more ways than one, more than you can count actually.

So it didn't take 125 years to get that way, it had a better start than most, and has only gotten more strict over the recent decades as computer statistics became mainstream.

Edit: Forgot to mention, there's no eminence. Nobody's name appears at the top of the document, and almost nobody (still living) ever appears within the text.

Further edit: I guess you could say that ASTM is a product of the Industrial Revolution, and there hasn't been an equivalent Academic Revolution yet.