[EGreg]

Your comment seems self-contradictory.

AI makes the fight against spam harder since it is harder to detect fakes. The group would only get more successful over time. The “AI detection” tools get worse over time and overhyped already anyway, as we learned from the guys running that sci fi submission site.

[ushtaritk421]

Right, but an org that accepts fake papers (now easily generated) gets outed easily and ignored by everyone.

Legitimate orgs with real review processes presumably sniff out the fake papers easily with their current procedures and don't publish them.

And I guess if AI gets good enough that an expert reviewer learns something novel by reading it then it deserves to get published.

[EGreg]

That’s exactly the point. What you previously assumed review processes could catch is no longer true. AI can now infiltrate pretty much anything, especially if there is a Generative Adversarial Network (which “AI detection tools” can be used to train at scale).

And if you say it “deserves to get published” then taken to its logical conclusion, human generated content in all fields and interactions will soon be dwarfed by AI content and interactions.

The issue is that the AI can be switched out after it’s infiltrated and taken over, or it can be gradually used to shift public opinion or organize any sort of coordinated attack. Heck, a reputational attack is easy to pull off at scale within 6 months via AutoGPT already, and it takes 1 button press.

First they’ll separate us, then they’ll herd us into echo chabers and cause our protests won’t be heard by anyone anymore amid all the AI glut.

It comes gradually, then all at once!