[SilverBirch]

I think the best explanation is to look at Google. Google's basic algorithm was that it could look how people organically interacted on the web and use that as a heuristic for quality - if lots of are linking to you, you're probably high quality and you'll appear at the top of google. But that started to break down, (a) because people were gaming that metric for "SEO" and (b) the internet centralized so the organic interactions started to disappear, and (c) because people stopped clicking through links from different sites - why do that when you can just google what you want! Google basically broke this metric by using it.

In the same way, AI is trying to generate text that looks like its training data, but if its training data is AI generated text then it's simply being taught to be more like itself. It slowly starts to work less like a human and more like whatever its own idiosyncrasies are. It's a larger sort of version of the hallucinations it has today. If 50% of all the text on the internet becomes some part AI generated, then a huge part of the training for the next generation of AI will be the shortcomings of the current iteration of AI. And this will get worse as non-AI content moves to exclude itself from training.