[simonw]

“AI companies avoid using AI data for training like the plague”

That’s not accurate. All of the big LLM training labs are leaning increasingly into deliberately AI-created training data these days. I’m confident that’s part of the story behind the big improvements for tasks like coding in models such as Claude 3.5 Sonnet.

The idea of “model collapse” from recursively training on AI-created data only occurs in lab conditions that very deliberately set up those conditions, from what I’ve seen. It doesn’t seem to be a major concern in real-world model training.