[Bjorkbat]

I'm not sure how much of a risk this is to LLMs in particular, but I feel like we're already seeing the impact on image AI models.

Even though they're getting better at generating hands that make sense and other fine details, you can generally tell that an image is AI generated because it has a certain "style". Can't help but wonder if this is partly due to generated images contaminating the training data and causing subsequent AI image generators to stylistically converge over time.

[astrange]

It's because the models don't have an optimal aesthetic policy. Which would be difficult, but if they did have one, it wouldn't matter how much bad input data you added during pretraining.