[Retric]

Journalists believing what you tell them says nothing about if the underlying work is actually significant.

The legacy of bad science being picked up is why this is a bad idea, even you personally don’t think it’s an issue the risk reward isn’t about just you.

[adjejmxbdjdn]

There are other quotes that do think it’s significant. Why do you think the more critical scientists are more correct?

> John Glass, a synthetic biologist at the J. Craig Venter Institute in La Jolla, Calif., who was not involved in the study. “It is dazzling that she has put these things all together,” he said.

> “We’re going to remember this moment,” said Roseanna Zia, a computational biologist at the University of Missouri who was not involved in the project.

Maybe you prefer scientists who put their money where their mouth is (even if that’s a conflict of interest)

> When Dr. Adamala showed SpudCell to Dr. Endy… (a synthetic biologist at Stanford University) … last year, he was so awestruck that he decided to help her found Biotic, the nonprofit organization intended to create a community of SpudCell researchers. “I’m pouring my life’s work into this,” Dr. Endy said.

[Retric]

> Why do you think the more critical scientists are more correct?

I am not qualified to make a judgement here, the point is following the process is better than jumping the gun on principle.

It literally doesn’t matter if it’s eventually considered groundbreaking research or not, jumping the gun is a bad idea.

[thaumasiotes]

> I am not qualified to make a judgement here, the point is following the process is better than jumping the gun on principle.

Why? The process is quite obviously net negative; we'd get better results with no process at all.

[Retric]

> we'd get better results with no process at all.

I’m going to give some advice that you probably won’t understand for years, but when you don’t find value in a process you’re missing something about what it’s doing.

A common shortcut is to look past who is making money to who is paying for that process and why they would want to pay for it.

[thaumasiotes]

In this case, the process is paid for by the government, and the reason they do it is that they wish to outsource the decision of which scientists on their payroll should be promoted.

If you're willing to stipulate their goals, it's easy to understand why they appreciate this system. But there is no benefit to other parties. As far as society is concerned, this is a big loss and an unforced error.

[Retric]

You’re mistaken in thinking it’s paid for by the government, though yes many governments are collectively a significant funding source they are far from the only funding source here.

The most critical function by far is it saves people doing research vast amounts of time. That includes people working at pharmaceutical companies, students, and non profits etc not just government employees. Thus why private colleges who don’t do cutting edge research as well as private labs etc still subscribe to such journals and thus fund the system.

This is a vast win for society. Could it be improved, sure, but you need to understand the value in order to build something that’s an actual improvement.

[derektank]

>The legacy of bad science being picked up is why this is a bad idea, even you personally don’t think it’s an issue the risk reward isn’t about just you

Who do you believe should be the gatekeeper here? Why can’t the scientist and the news outlets be trusted to make the decision about whether to publish or not themselves? Why can’t the general public be trusted to evaluate the quality of the news outlets they read?

[Retric]

> Why can’t the scientist and the news outlets be trusted to make the decision about whether to publish or not themselves? Why can’t the general public be trusted to evaluate the quality of the news outlets they read?

Because the scientists involved and reporters manifestly do a bad job about picking what is or isn’t groundbreaking and more importantly have various incentives to hype things up.

CERN scientists with the whole FTL neutrino particle were actively skeptical of the results and still held a press conference on the topic. As to reporters, you’re welcome to go through the published stories about the topic and notice how rarely getting the distance wrong was brought up even when the scientists involved where skeptical. https://en.wikipedia.org/wiki/2011_OPERA_faster-than-light_n...

The general public is utterly incompetent at judging science. Homeopathy is the tip of the iceberg of ignorance.

[derektank]

You didn’t answer the actually interesting question I posed, who should be the gatekeeper instead?

Yes, yes, journalists and scientists have bad incentives and the general public is dumb. You’re not exactly setting the world on fire with that observation. The problem is that there is no better alternative. Any conceivable gatekeeper to scientific knowledge will be no smarter than the research scientists producing the results and will certainly have problematic incentives of their own. And a gatekeeper will also lack the local knowledge that might determine whether the information might helpful or harmful to the potential reader.

[Retric]

Programming reinvented a similar system of peer review before commits. It’s not that the reviewer is more knowledgeable about the bits of the system being changed that makes this work it’s that they have enough expertise to understand what’s involved and a new set of eyes on the problem.

In science having multiple journals acts as a safety valve here, but the underlying principle is very similar. As much as some people bad mouth it, peer review is a very low hurdle before publication that still catches a great number of mistakes.