[wand3r]

> What if we're currently in peak LLM? The moment in history where ~none of the content used to train them, and to have them operate on is aware of its LLM consumers, but from now on everything will be, and the quality of LLMs will slowly decrease?

Having read the authors summary of what they mean by "Peak LLM" I do agree to an extent. As reams of shitty wordpress sites pollute the internet regurgitating GPT prompts and people take action to dissuade indexing the AVERAGE data quality will go down.

However, unlike Google which has a perverse incentive to fix blogspam and SEO bullshit and improve search, as worse search means more searches, means more money; LLMs are greatly incentivized to improve. Additionally, there are archives of the past web which should backstop most non-current answers.

It's definitely a REAL consideration for sure that the data and inputs will get fucked up, but I suspect it will be a solvable problem.

[vouwfietsman]

This is only true if, like now, the entities controlling LLMs are research centres. I think its likely the future owners of LLMs have similar incentives to google to monetize the project.