Hacker News new | ask | show | jobs
by silisili 604 days ago
What's it called when bots start learning primarily from other bots and get stuck in a loop, no longer acquiring any real new intelligence?
1 comments

Model collapse. "No longer acquiring any real new intelligence" would actually be a big breakthrough, I think - with current techniques we don't just stop improving, but start degrading. If LLMs are blurry jpegs of the entire corpus of human knowledge, then it's easy to imagine what happens when you start making a jpeg from a jpeg.