They are going to make it a thousands times worse.
It wasn't perfect before, but it at least took some time to fake a paper. The problem is now people can produce a very plausible looking completely fake paper in minutes. Peer review is in the process of completely collapsing, in fact I think it's already basically done.
The only way this might fix things is if we require all papers are completely reproducable (that doesn't help in subjects like biology of course. They can still provide all the experimental data in the rawest format possible which doesn't break any laws).
In some fields like comp sci, when code isn't given but the paper describes the approach, LLMs do help with the reproducibility crisis: you can ask it to reproduce the result through reimplementation by reading the paper.
If it fails you may have to double check it did properly reimplement it, but if it succeeds you do get a reproduction.
I'm actually quite excited for when (if) the models get good enough to start replicating compsci papers. I'd love it if there was a system which calculated a reproducibility score per-lab or per-researcher, which I could look up alongside their citation count.
I want to see who did the hard work properly, and who focused on publishing with concealed details.
It seems to me that LLM's could massively improve reproducibility issues if journals would require that the papers be reproducible by model X using a standardized prompt in < N minutes, etc...
Por que no los dos? Scientific review times are up, it’s harder to find reviewers, and many reviews are AI generated anyway. Auto-generated research publications will arguably make the replication crisis worse, because there will be more slop to clog up the review system, and these papers will presumably be just as (if not more) not reproducible than human written science
it could also be said that scientific interpretation is suffering from a framework crisis. the scientific convention of experiment, is the test of an hypothesis, as a logical construct.
repetition of materials and methods toward reproducibility, holds far less wieght than multiple variants of process designed to test a common hypothesis resulting in agreement.[null, or failure to null]
For a while now there has been very little incentive for providing these alongside the paper, and I don't see why exactly 'AI' would change this. I could even see how making it vague to be harder to test with LLMs could be profitable for citation hackers.
You can imagine using AI agents to tag papers that don’t have code or similar work attached and just filtering them out.
The Chinese open source community has made a lot of incentive to make research reproducible for example. The most reproducible works from I.e. deepseek get widely cited and adopted.
I don’t think we can just say “AI” and it’s fixed but with deliberate effort there’s reason to be optimistic.
Unless reviewing becomes more profitable than publishing, anything that makes both easier will drive one up far more than the other. And it is difficult to conceive of something that would make reviewing much easier without making publishing much easier.
Just as a counterpoint ML and AI research has become much more reproducible over time. I feel like this is relevant because ML / AI researchers are huge power users of AI tools.
Between 2016 and 2021 the share of ML/ robotics/ AI researchers being reproducible (ie contianing code and similar instructions to reproduce) doubled [1].
The major US labs have gone largely closed source (I.e. they no longer publish frontier research) but the Chinese ecosystem has incredibly reproducible code.
This is field dependent obviously but I think it atleast gives reason to be optimistic.
Yes people will churn out fake slop research, but it feels like that can be categorized and then ignored.
That's good to hear about ML and AI research, but most research is not based on computers and so would require laboratory setups to reproduce. Not only is trying to reproduce such findings (beyond what is effectively a sanity check) through simulations a lost cause, if AI can reproduce such research it would be capable of doing such research itself... in which case it would be far more fruitful to use AI to do further research.
This thread is about a product based on fully reproducible research though so I feel like we should stay grounded. Claude science is meant to be used in the context of reproducible science research, there is a decent reason to not be cynical on future research being reproducible.
> if AI can reproduce such research it would be capable of doing such research itself
Well there is a big distinction between research validation and research generation, it is generally much easier to verify that a math proof is true or false than to find a truly novel proof.
But yes in the long run I’d think AI will be doing tons of research and it will by default reproducible. So maybe we’re aligned after all?
It wasn't perfect before, but it at least took some time to fake a paper. The problem is now people can produce a very plausible looking completely fake paper in minutes. Peer review is in the process of completely collapsing, in fact I think it's already basically done.
The only way this might fix things is if we require all papers are completely reproducable (that doesn't help in subjects like biology of course. They can still provide all the experimental data in the rawest format possible which doesn't break any laws).