[sirwhinesalot]

Current LLMs are already trained on the entirety of the interwebs, including very likely stuff they really should not have had access to (private github repos and such).

GPT-5 and other SoTA models are only slightly better than their predecessors, and not for every problem (while being worse in other metrics).

Assuming there is no major architectural breakthrough[1], the trajectory only seems to be slowing down.

Not enough new data, new data that is LLM generated (causing a "recompressed JPEG" sort of problem), absurd compute requirements for training that are only getting more expensive. At some point you hit hard physical limits like electricity usage.

[1]: If this happens, one side effect is that local models will be more than good enough. Which in turn means all these AI companies will go under because the economics don't add up. Fun times ahead, whichever direction it goes.

[Macha]

I wonder what the programming equivalent of the AI yellow filter for images is going to be