[probably_wrong]

I recently desk-rejected a paper where every single citation in its Introduction was hallucinated. That means that the entire connection between what the author(s) did and how it relates to existing research was simply made up. I've never seen this happening before AI but now there's at least one paper in every cycle pulling something similar.

My problem therefore is: we are seeing more and more papers written with tools that are known to make up facts, citations, and even entire papers. And the number of papers has increased, too. I therefore see it less as "people are being more productive" and more "people are releasing bad science much faster than we can keep up with".

[guywithahat]

Yeah but that's an issue with the researcher putting out a bad paper, and it suggests you'll have to reject more papers. We wouldn't ban email because many of the emails are spam, it just means we need new tools to filter out junk. AI will allow researchers to be more productive all together and take less time to publish a paper, which is good.

[probably_wrong]

The problem with spam is that it's not a technology like AI is. So I suggest taking cars instead.

Cars have plenty of advantages, and yet no one would say "the number of pedestrians killed by cars is rising, but that's an issue with the drivers". In fact, the opposite is true: from fines and school zones to speed bumps and bollards, we have accepted that cars bring structural problems with them that cannot be solved at the driver level alone.

> we need new tools to filter out junk

Agreed, but if my office suddenly was flooded with garbage my first thought wouldn't be "I need more, bigger trash cans" but rather "who brought all this junk here and why?". To simply assume that the garbage is a sudden natural phenomena that I have to live with seems, at the very least, unfair.

[lelanthran]

> Yeah but that's an issue with the researcher putting out a bad paper, and it suggests you'll have to reject more papers. We wouldn't ban email because many of the emails are spam, it just means we need new tools to filter out junk. AI will allow researchers to be more productive all together and take less time to publish a paper, which is good.

It's a signal:noise ratio thing. If 1 out of every 1000 AI-written papers are bad, it makes sense to put in a filter that auto-rejects any paper that has AI tells.

After all, if that 1 researcher was any good, he wouldn't have used AI to write the thing in the first place.

Publishing was always about getting past the filters. There's one more filter - "AI-generated content" - so do what you have to to get past it. IOW, write your own paper.

[verdverm]

While working on my PhD, there was one guy in the field that would publish new, better results to Springer every time someone improved upon benchmarks. No code, no data, no conference publications, paywall restricted, unverifiable, always the best.

I am almost certain he was "hallucinating" the results. This was in the 2010s

There are well known issues in academic publishing, though I imagine it has become much noisier like open source