[JustinCS]

Even as AI generates more writing and code, we still have a way of ranking quality: Good writing and successful projects tend to get more popular and prominent. This selection can allow LLMs to continue to improve. They get a huge flow of slop, but they generate based on the patterns correlated with better quality. The model developers can also develop better ways to curate the input data themselves and keep the slop at bay. It's not a guaranteed or trivial mechanism, but I don't think we need a new breakthrough either.

[ttiurani]

Maybe.

As a counterpoint: isn't popularity of a library more a metric of API convemience than actual code quality?

And isn't popularity of an essay more about how it conforms to existing beliefs than the quality of the thinking?

[JustinCS]

Those are good points and that's why progress is not guaranteed or trivial, just plausible.