[ajsnigrutin]

Now imagine all that low quality AI slop is being posted online and a new generation of AI will "learn" from it, output it's own version of AI slop, that will eventually end up online again for a new generation of AI to "learn" from.

Something, something, idiocracy comes to mind.

[bitwize]

This leads to a well-documented phenomenon known as model collapse. You know how if you blur and sharpen an image repeatedly you eventually end up with just a rectangle of creepy, wormy spaghetti lines? You lose information on each blur, and then ask it to reconstitute the image with less information on each sharpen, until there's nothing recognizable left.

Training a model is like the blur and generating from that model is like the sharpen. Repeat enough times and enough information is lost that you're just left with "wormy spaghetti lines"—in an LLM's case, meaningless gibberish that actually pretty closely resembles the glitchy stuff said by the cores that fall off GLaDOS in Portal. I dunno, you read the paper and be the judge:

https://www.nature.com/articles/s41586-024-07566-y

To jump to the last output sample, C-f Gen 9

Of course you may be talking about the human aspect of this. Gods willing, we'll realize that our LLMs are spewing gibberish and think twice about putting them in all the things, all the time. But the scenario I fear isn't Idiocracy—it's worse: a community of humans who treat the gibberish as sacred writ, Zardoz style.

[Nevermark]

> Something, something, idiocracy comes to mind.

So, confirmation? They are catching up quickly!

The actuality is, anyone with pre-slop data still has their pre-slop data. And there are endless ways to get more value out of good data.

Bootstrapping better performance by using existing models to down select data for higher density/median quality, or leverage recognizable lower quality data to reinforce doing better. Models critiquing each other, so the baseline AI behavior increases, and in the process, they also create better training data. And a thousand more ways.

Managed intelligently, intelligence wants to compound.

The difference between human and AI idiocracy, is we don't delete our idiots. I am not suggesting we do that. But maybe we shouldn't elect them. Either way, that is one more very steep disadvantage for us.

[livinglist]

The AI centipede