[Taek]

It means LLMs might self-improve beyond the point where we can comprehend how intelligent they are. An LLM with 10x the capabilities of all human brains combined is indistinguishable to a human with one that has 100x the capabilities of all humanity combined, effectively making it possible for there to be "no peak"

[ihatepython]

I don't think you understand how LLMs work

[Taek]

I think you underestimate how intelligent LLMs are. If the training data only explains a certain concept in French, the LLM will nonetheless be able to tell you about that concept in any other language it has proficiency in. There's clearly a lot more going on under the hood than just a sophisticated markov chain.