[simonw]

Personally I don't believe in model collapse. Has anyone demonstrated it occurring in the wild, outside of the tiny set of papers that deliberately caused it to happen?

I think model collapse gets talked about so much because it is irresistible schadenfreude. The idea of models eating their own tails in a way that leads to their inevitable demise is captivating to a lot of people, especially AI skeptics.

[pama]

I agree. A partial counterexample is the RL training loop on verifiable tasks, which uses the model in a loop to generate training data. Another one is the cleanup/prioritization of the pretraining data using earlier models.

More generally, a lot of ideas have been speculated based on very tiny models in controlled settings and they didnt pan out in real LLMs. There probably exists a minimal compute threshold for overcoming generalization traps.

[marbro]

Carbon-based model collapse is known as groupthink and happens constantly.