[ushtaritk421]

Feels like AI makes the fight against paper mills easier. A group could submit AI-generated papers to different places and create a public index based on how successful they are at getting their nonsense papers accepted.

[bagels]

https://en.wikipedia.org/wiki/Sokal_affair

[SpaceBuddha]

"Later, after Sokal revealed the hoax in Lingua Franca, Social Text's editors wrote that they had requested editorial changes that Sokal refused to make, and had had concerns about the quality of the writing: 'We requested him (a) to excise a good deal of the philosophical speculation and (b) to excise most of his footnotes.' Still, despite calling Sokal a 'difficult, uncooperative author", and noting that such writers were 'well known to journal editors', based on Sokal's credentials Social Text published the article in the May 1996 Spring/Summer 'Science Wars' issue"

[noobermin]

That even makes it worse, I somehow didn't know this part of it.

[BeetleB]

Sokal did not publish in a peer reviewed journal.

[EGreg]

Your comment seems self-contradictory.

AI makes the fight against spam harder since it is harder to detect fakes. The group would only get more successful over time. The “AI detection” tools get worse over time and overhyped already anyway, as we learned from the guys running that sci fi submission site.

[ushtaritk421]

Right, but an org that accepts fake papers (now easily generated) gets outed easily and ignored by everyone.

Legitimate orgs with real review processes presumably sniff out the fake papers easily with their current procedures and don't publish them.

And I guess if AI gets good enough that an expert reviewer learns something novel by reading it then it deserves to get published.

[EGreg]

That’s exactly the point. What you previously assumed review processes could catch is no longer true. AI can now infiltrate pretty much anything, especially if there is a Generative Adversarial Network (which “AI detection tools” can be used to train at scale).

And if you say it “deserves to get published” then taken to its logical conclusion, human generated content in all fields and interactions will soon be dwarfed by AI content and interactions.

The issue is that the AI can be switched out after it’s infiltrated and taken over, or it can be gradually used to shift public opinion or organize any sort of coordinated attack. Heck, a reputational attack is easy to pull off at scale within 6 months via AutoGPT already, and it takes 1 button press.

First they’ll separate us, then they’ll herd us into echo chabers and cause our protests won’t be heard by anyone anymore amid all the AI glut.

It comes gradually, then all at once!

[distant_hat]

Reviewers are already overloaded with reviewing all kinds of papers and aren't paid for it. To increase their workload is an absolute worst outcome.