[simplicio]

Seems the fastest way to clear this up would just be for the original team to send a sample to some other scientists for testing. That would solve any issues regarding other teams reproducing the material incorrectly, and at a first pass, what everyone cares about is whether its a superconductor at room temp, not how to make it.

[peyton]

> Once the findings are published in a peer-reviewed journal, which Kim says is in the works, he will support anyone who wants to create and test LK-99 for themselves. [1]

Sounds like that’s the plan once they’ve secured the Nature paper.

[1]: https://www.newscientist.com/article/2384782-room-temperatur...

[p-e-w]

Sounds like they have it the wrong way round. Considering the uncertainty surrounding all of this, and the initial problems with replication, why would Nature publish this before it has been independently replicated?

Not sure what they're worried about here. They already have priority, and if this replicates, every journal on Earth will be fighting over the privilege to publish it.

[Maxion]

Because that’s the scientific method? It would be entirely infeasible for them to require someone else to test the sample.

I don’t think you’ve been in academia?

[p-e-w]

> Because that’s the scientific method?

The world's leading science journal publishing grand claims without verification, in the face of mounting evidence that the claims might be incorrect, is most certainly not "the scientific method".

> It would be entirely infeasible for them to require someone else to test the sample.

What nonsense. Why would that be infeasible?

[szundi]

Actually this is the scientific method. Anything else just makes publishing groundbraking results harder and depends on some people’s (faulty) judgements.

[mort96]

Publishing incorrect results should be hard, especially if they would be ground breaking if true.

[physicsguy]

Nature basically publishes the sensational claims papers. That's their model, anything "high impact" goes there. It's then on other scientists to publish work refuting it or proving it's not correct. It's an issue that the refutations tend to go in lower impact journals, but it's the way it's works.

[epups]

Extraordinary claims require extraordinary evidence.

But seriously, why can't they ship their sample somewhere?

[SiempreViernes]

They are busy trying to characterise it properly and making sure they understand what they have created. At least that's what they ought to be doing if this isn't a big scam.

[ethbr0]

> why would Nature publish this before it has been independently replicated?

Because it's Nature?

We're not talking about Organic Syntheses [0], which has replicated every paper before publishing since 1921.

[0] https://www.orgsyn.org/

[peyton]

I guess don’t hate the player. They’ve got a floating rock. What would materially affect conclusions?

[zarzavat]

Pride. They don’t want to be remembered for some incomplete Arxiv preprint. They want to be remembered for the finished paper in a top level journal.

[WillAdams]

Most likely the peer review process will include replication.

[tnecniv]

Normally (at least in my field), that’s beyond the scope of peer review in the traditional sense. Peer review typically just involves review of just the material submitted to the journal (manuscript, data, maybe code depending on its size and complexity). Depending on the experiment, reproducing a result may require resources, equipment, or skills unavailable to the reviewer or that cannot be acquired in a reasonable amount of time needed for the review.

Peer review serves basically as a smell test where capable experts certify that the paper is novel, interesting, properly justifies its claims, and contains enough detail to reproduce the results. It is not uncommon for only a few labs to contain all of the expensive, specialized equipment to reproduce an experiment, and getting that experiment working (if it is reproducible) can take a significant time commitment from multiple individuals in a lab. Also keep in mind that reviewers are normally not paid for their services, journals typically want at least two independent reviewers (who they pick, sometimes with input from the author to narrow down who has the background to perform the review), and going from submission to publication is typically a many-month process without the high standard of reproduction.

Reproduction happens in the manner that is going on right now. If your publication makes waves, people will want to build on your result and either reproduce it as a first step or conduct a related experiment that adds evidence to support the hypothesis. This way is far more practical, but the devious thing is that, if you lie and manipulate your data, some poor grad student might waste two years of their PhD trying to get your thing to work.

Having journals or some independent body fund reproduction themselves runs into both capital challenges and the problem of attracting a large enough number of capable scientists to cover every discipline who would only want to work on reproduction full time. As nice as that would be, I’m not sure such a venture would work in the real world.

[jacquesm]

That's true but if a reputable journal publishes this before that verification is in and it ends up being mistaken or even a hoax or fraud then their reputation will be dinged so with the stakes this high given the material and the visibility there is a fair chance that they will indeed wait for independent replication before committing to publishing it. But I don't see it happening during the peer review process itself, it will happen simply because another lab tries the same and gets lucky (or not, in which case the whole thing will blow over).

[phanimahesh]

The journals reputation will be dinged if it comes to light that they have not adequately performed their due diligence. Which, as has been explained above, does not include replication.

If you require replication for publication practically nothing will get published ever. Once published, others will attempt to replicate, and it is the authors' reputation on the line if nobody manages to replicate it following the paper and consulting with the authors.

[RF_Savage]

How can you replicate it before the paper with the stuff to replicate is published?

[lovich]

Why would Nature have published the production method of a random material if they didn't have experimental evidence of novel behavior?

[cosmojg]

They do that every now and then. Peer review rarely, if ever, involves any sort of replication.

[lovich]

I’m asking on a legitimate basis. I am not involved in the science world and only know about Nature as a publication from high school and college. The way Nature was described to gave me the sense that you weren’t going to have a paper published unless you had some surprising results, and that didn’t include the null hypothesis.

If that’s not the case for this publication then I would appreciate if someone showed me how they changed in the past decade

[michaelt]

They publish surprising and high-profile results, yes.

However, there's also an informal crazy claims filter. If your paper is so surprising they think it might be made up or wrong, they might ask for extensive revisions prior to publication.

So they won't publish my paper saying the moon is made of cheese no matter how surprising such a finding might be.

[bluGill]

Easy to say, but this is hard. "I got a box of dust from you". Now was this the lab sending dust because they know it doesn't work, and want to hide their tracks; the receiving lab destroying the sample because they want their own discovery to win a Nobel prize; or just regular managed in shipping? You should be able to think of other reasons that an unethical lab on either end might want to ensure the samples are destroyed instead of tested.

The above can be managed, but it can't be done over a weekend. The lab has stated they are willing to send their working samples out to other labs. However to ensure it arrives safely (avoiding issues like the above) means that you shouldn't even be concerned until two months have passed, and accept reassurance for a few months after that.

[regular_trash]

This assumes the chemical structure is stable, which may not be the case. If the material is truly not superconductive at room temperature, it will take a long time to gather enough diverse experimental evidence to build a scientific consensus. A good rule of thumb is probably 3 months to reproduce, 6 months to discredit.

[faraggi]

why is it slower to discredit than to reproduce?

[roywiggins]

If I can replicate a result, that's fairly straightforward: I replicated it!

If I can't, that doesn't mean the result was necessarily wrong; maybe I just messed up, or got unlucky and the result only happens 20% of the time. You might need several independent failed replications to start to be confident that the original result was definitely wrong.

It's like saying: there's a buried treasure in this acre. If I find it on my first pass through with a metal detector, job done. If I don't find it on my first pass, I'll probably need to make several more passes, maybe bring in fancier equipment and so on, before I could be pretty confident that it's not there.

[Remnant44]

Someone claims that a human can run a 5 minute mile.

To prove it true, you just need to show a human who can run that fast. To provide it fast, you have to accumulate incontrovertible evidence that it's simply beyond us...

[Toutouxc]

For those firing up Google: The mile run record has been sitting at 3:43 for over two decades now.