[wddkcs]

Most professionals didn't think we were close to surpassing human capability in chess, go, or dota, until after it happened. I've seen little evidence of expert domain knowledge improving AI forecasting ability, if anything it seems the experts are often late to the party.

Besides expert consensus, is there any other actual argument against LLMs achieving generalizability?

[mianos]

> Besides expert consensus,

Well there are solid technical reasons, as described in the video. One of them is based on that these models are 'pre-trained' and AGI may be a result of a more dynamic knowledge base that can change more than just the local context and update the model, as our brain does.

Andrej also suggests that an attribute of a more advanced AI would have the ability to ask it to spend longer thinking to get a better answer, like a chess engine.

This said, expert consensus is probably the best answer we have. It's not like the consensus of a bunch of youtube vids and articles that only exist for getting clicks. These experts are famously sharp. I have done his course video series (it took a huge effort, even though he is an amazing lecturer) and had existing python and linear algebra experience and I understand his argument.

[wddkcs]

>these models are 'pre-trained' and AGI may be a result of a more dynamic knowledge base

Why couldn't the knowledge base be used in conjunction with the LLM? As the GP said, why can't LLM's gain sentience or be finagled into sentience with a wrapper'. The Knowledge base you're describing is the wrapper.

>Andrej also suggests that an attribute of a more advanced AI would have the ability to ask it to spend longer thinking to get a better answer, like a chess engine.

This is another method that is already being deployed with LLMs. So the question stands, why won't LLMs be the foundation for nearing AGI?

For my money, LLMs likely are that base. AI Experts are either too shy from the memory of AI winters past to see the nose on their faces, or too busy developing paradigm breaking models to care. Regardless of what Chomsky or any other 'expert' says should be possible, the practical results of LLM growth are literally speaking for themselves.

Maybe we should have suspected a 'large language game' to be the catalyst for AGI from the start. Was human intelligence truly general before we developed language? Could it be general without it?