[TeMPOraL]

Also if they're solving problems to help LLM training in their domain, that's actually pretty useful contribution to science - and definitely more directly useful than the work that dominates actual research, i.e. chasing grants instead of researching.

[LeCompteSftware]

"that's actually pretty useful contribution to science"

Why? Serious question. Surely the only people using the LLM for such specific STEM domains are the exact same people who are "chasing grants instead of researching." Certainly I can see how training an LLM on this stuff can help automate the process of grant-chasing, and maybe OpenAI can expand their homework cheating business to graduate schools. But I do not see how this stuff helps honest researchers, except a bit around the margins (e.g. perhaps Claude isn't so good at the Perl used in bioinformatics, that's a use case justifying some RLHF from a PhD).

It really seems like the main utility of this stuff is getting a higher score on Humanity's Last Exam and showing the customers/investors that actually Opus 4.9 is 2% smarter than GPT 5.5. Separately there are AlphaProof/etc-style LLMs for solving real research problems in math and CS, but those techniques don't even work for theoretical physics, let alone biology.

[TeMPOraL]

LLMs are actively used in research all the time, they help with finding and processing existing knowledge, forming and testing hypotheses, analyzing data, writing software, brainstorming, and countless other tasks that form actual research work, as distinct from "grant chasing" and "publishing papers", in which they help, too.

(I mean, OpenAI released GPT-Rosalind just yesterday, and - surprise - it's not meant for chasing grants.)

It's not 2023 anymore, it's 2026. LLMs are good enough to be useful. They have been for at least a year, and they keep getting better. You need to be living under a rock for the past few years to not notice that.

[LeCompteSftware]

This doesn't even slightly answer my question. The incredibly frustrating thing about the AI discussion is the refusal to consider actual evidence because of shifting targets. In 2026 there is evidence that 2024 LLMs did enormous damage to scientific research in 2025: hallucinated citations, hallucinated experiements, an onslaught of unreadable prose, etc etc. But we can't talk about that, can we? That's old hat, everybody knows 2024 LLMs were stupid and useless. Instead we have to discuss our vibes about 2026 LLMs, and maybe in 2028 we'll be able to tell whether or not our vibes were correct.

[TeMPOraL]

LLMs couldn't do any damage with hallucinated citations - on the contrary, this is only ever a problem for people so clueless and uncaring that they didn't even bother reading what LLMs wrote for them. Hallucinated citations are evidence of fraud or level of uncaring unbecoming a scientist, or any professional on that matter.

[LeCompteSftware]

"LLMs couldn't do any damage with hallucinated citations"

If you're saying stuff like this with a straight face then you are clearly not a scientist and you don't know what you're talking about. In 2021 there were maybe 10 papers with fictional citations. Even the publication mills at least linked to other junk papers. Now there are hundreds of thousands of papers with dishonest and useless bibliographies. This is because LLMs are an unbeatable force multiplier for dishonest and useless scientific work.

I am sure some legitimate academics are getting real use out of them. I am also sure that the net effect of LLMs on science is enormously negative, and it will take decades to fix the mess.

[TeMPOraL]

Unlike the industry, science has actual standards of conduct, which puts it in a unique position to fix it quickly - if only the journals were doing the one job they have.

Hallucinated citations == strong evidence of scientific fraud. Name and shame and don't publish.

Alas, what's happening only shows the emperor has no clothes. If anyone slept through the replication crisis, they surely can't ignore it now. Can't really blame LLMs for lighting up the structural corruption of scientific process for everyone to see.

If anything, it's doing us a favor - if the journal gatekeeping and peer review can't handle people putting literal, obvious bullshit in their papers today, think what else they aren't handling either, and for how long this has been the case.

[trevithick]

Still bad for the scientists. They get little money and zero recognition.

[TeMPOraL]

Right. They get to contribute something useful and be paid for it, which is better than nothing, but it's sad that their talent is being wasted.

[philwelch]

They already didn’t get money or recognition.