[weakfish]

You’re assuming that progress will continue at the same rate (it’s plateaud a bit as far as I can tell, I can look for sources tomorrow but am on phone now :))

& I think it’s fairly demonstrated that the industry isn’t profitable for training ever increasingly huge models, which get exponentially more expensive each iteration

[galaxyLogic]

I wonder, LLMs have progressed at great speed, because they have learned to duplicate what humans did and recorded in the documents and source-code that became the training materials for the LLMs. But now that humans don't write any more code, there is no new learning materials for the LLMs. There are no new things for them to learn. Therefore I expect they will plateau, the progress is no longer so fast.

And maybe we'll get something like a Mad Cow Disease, when AI "eats its own brains". To avoid that from happening we will need continued human inputs, and review.

[inigyou]

They are now paying full-time programmers to write new training data.

[galaxyLogic]

In other words humans are teaching the AI.