[calny]

> if LLM-generated content outpaces human-generated content, the useful data proportion will diminish, and in conjunction with LLM content optimization it will become exponentially harder to find useful new bits

While possible and concerning, this isn’t inevitably true. To take the optimistic view, LLMs can be more than simple regurgitation machines, and can create new insights from existing knowledge. Novel/useful LLM content that’s created today can be training input for future LLMs to derive even further new insights.

[te_chris]

But there’s almost no way to tell at scale which part of the training set is your novel, useful stuff and which is pure bullshit.