Hacker News new | ask | show | jobs
by jbenjoseph 1206 days ago
But even so, the human picks the prompts and only publishes the AI outputs they think read nicely. There is information gain even in that.
1 comments

At the moment that's probably true, but is it guaranteed to remain so?
Even so, I don't think there is any evidence that LLM performance degrades when it is trained on its own output, and there is no intuitive reason it should.
Why not? Training a model destroys information.
I have seen no evidence for that, only the opposite: https://arxiv.org/abs/2210.11610

Intuitively, training a simple enough linear statistical model with its own output should be a NOP. But LLMs are anything but simple models, so I think the non-linearities may be synthesizing new useful information. Similarly to how all of maths can be synthesized from a few basic axioms with enough intelligence or computation.