I remember similar notions in his novel Fall — where the internet is so full of fake news and sponsored content that people need additional services to filter out useful information
Oh cool, I haven't read Fall but it's similar in Anathem
The reticulum (aka internet) is filled with software generated crap. Initially most of it is blatant and easy to filter out, like an over the top "but cheap v14gra" spam email.
Eventually the companies that make the crap filtration software get into an arms race with each other, they realize if they can generate their own crap that their competitors don't detect, it'll give them a competitive advantage. The reticulum becomes filled with "high quality" crap, eg text that's ALMOST correct but wrong in subtle ways. Imagine a wiki article about pi where everything is correct except the 9th digit, or an op-ed with slightly flawed logic.
Eventually crap goes beyond text, and the reticulum starts to see deepfaked images and videos that parallel news articles. One of the jobs of the ITA is to try and find the signal in all the noise.
> Early in the Reticulum—thousands of years ago—it became almost useless because it was cluttered with faulty, obsolete, or downright misleading information,’ Sammann said.
> ‘Crap, you once called it,’ I reminded him.
> ‘Yes—a technical term. So crap filtering became important. Businesses were built around it. Some of those businesses came up with a clever plan to make more money: they poisoned the well. They began to put crap on the Reticulum deliberately, forcing people to use their products to filter that crap back out. They created syndevs whose sole purpose was to spew crap into the Reticulum. But it had to be good crap.’
> ‘What is good crap?’ Arsibalt asked in a politely incredulous tone.
> ‘Well, bad crap would be an unformatted document consisting of random letters. Good crap would be a beautifully typeset, well-written document that contained a hundred correct, verifiable sentences and one that was subtly false. It’s a lot harder to generate good crap. At first they had to hire humans to churn it out. They mostly did it by taking legitimate documents and inserting errors—swapping one name for another, say. But it didn’t really take off until the military got interested.’
> ‘As a tactic for planting misinformation in the enemy’s reticules, you mean,’ Osa said. ‘This I know about. You are referring to the Artificial Inanity programs of the mid-First Millennium…’”
This sounds more inspired by trap streets than AI data pollution, though I suspect the people trying to fight generative AI will eventually resort to trap streets anyway, and wind up producing the same outcome.
The reticulum (aka internet) is filled with software generated crap. Initially most of it is blatant and easy to filter out, like an over the top "but cheap v14gra" spam email.
Eventually the companies that make the crap filtration software get into an arms race with each other, they realize if they can generate their own crap that their competitors don't detect, it'll give them a competitive advantage. The reticulum becomes filled with "high quality" crap, eg text that's ALMOST correct but wrong in subtle ways. Imagine a wiki article about pi where everything is correct except the 9th digit, or an op-ed with slightly flawed logic.
Eventually crap goes beyond text, and the reticulum starts to see deepfaked images and videos that parallel news articles. One of the jobs of the ITA is to try and find the signal in all the noise.