[sublinear]

> The architecture might just be wrong for AGI. LeCun’s been saying this for years: LLMs trained on text prediction are fundamentally limited. They’re mimicking human output without human experience.

Yes, and most with a background in linguistics or computer science have been saying the same since the inception of their disciplines. Grammars are sets of rules on symbols and any form of encoding is very restrictive. We haven't come up with anything better yet.

The tunnel vision on this topic is so strong that many don't even question language itself first. If we were truly approaching AGI anytime soon, wouldn't there be clearer milestones beforehand? Why must I peck this message out, and why must you scan it with your eyes only for it to become something else entirely once consumed? How is it that I had this message entirely crystalized instantly in my mind, yet it took me several minutes of deliberate attention to serialize it into this form?

Clearly, we have an efficiency problem to attack first.

[hackinthebochs]

>Yes, and most with a background in linguistics or computer science have been saying the same since the inception of their disciplines

I'm not sure what authority linguists are supposed to have here. They have gotten approximately nowhere in the last 50 years. "Every time I fire a linguist, the performance of the speech recognizer goes up".

>Grammars are sets of rules on symbols and any form of encoding is very restrictive

But these rules can be arbitrarily complex. Hand-coded rules have a pretty severe complexity bounds. But LLMs show these are not in principle limitations. I'm not saying theory has nothing to add, but perhaps we should consider the track record when placing our bets.

[sublinear]

I'm very confused by your comment, but appreciate that you have precisely made my point. There are no "bets" with regard to these topics. How do you think a computer works? Do you seriously believe LLMs somehow escape the limitations of the machines they run on?

[ACCount37]

And what are the limitations of the machines they run on?

We're yet to find any process at all that can't be computed with a Turing machine.

Why do you expect that "intelligence" is a sudden outlier? Do you have an actual reason to expect that?

[RandomLensman]

Is everything really just computation? Gravity is (or can be) the result of a Turing machine churning away somewhere?

[ACCount37]

It might be.

https://xkcd.com/505/

[keernan]

>We're yet to find any process at all that can't be computed with a Turing machine.

Life. Consciousness. A soul. Imagination. Reflection. Emotions.

[ACCount37]

Again: why can't any of that run on a sufficiently capable computer?

I can't help but perceive this as pseudo-profound bullshit. "Real soul and real imagination cannot run on a computer" is a canned "profound" statement with no substance to it whatsoever.

If a hunk of wet meat the size of a melon can do it, then why not a server rack full of nanofabricated silicon?

[keernan]

For the same reason you don't sit and talk with rocks. Nobody understands how it is that wet meat can do these things but rocks cannot. And a computer is a rock. As such, we have no idea whether all the hunks of wet meat in the world can figure out how to transform rocks into wet meat.

[hackinthebochs]

What do you think these in principle limitations are that preclude a computer running the right program from reaching general intelligence?

[ACCount37]

Why would language restrict LLMs?

"Language" is an input/output interface. It doesn't define the internals that produce those inputs and outputs. And between those inputs and outputs sits a massive computational process that doesn't operate on symbols or words internally.

And, what "clearer milestones" do you want exactly?

To me, LLMs crushing NLU and CSR was the milestone. It was the "oh fuck" moment, the clear signal that old bets are off and AGI timelines are now compressed.

[AlexandrB]

Language massively restricts LLMs because there's no way to create novel concepts while limited to existing language.

Humans create new words and grammatical constructs all the time in the process of building/discovering new things. This is true even in math, where new operators are created to express new operations. Are LLMs even capable of this kind of novelty?

There's also the problem that parts of human experience are inexpressible in language. A very basic example is navigating 3D space. This is not something that had to be explained to you as a baby, your brain just learned how to do it. But this problem goes deeper. For instance, intuition about the motion of objects in space. Even before Newton described gravitation every 3 year old still knew that an object that's dropped would fall to the ground a certain way. Formalizing this basic intuition using language took thousands of years of human development and spurred the creation of calculus. An AI does not have these fundamental intuitions nor any way to obtain them. Its conception of the world is only as good as the models and language (both mathematical and spoken) we have to express it.

[ACCount37]

> Its conception of the world is only as good as the models and language (both mathematical and spoken) we have to express it.

Which is pretty damn good, all things considered.

And sure, training set text doesn't contain everything - but modern AIs aren't limited to just the training set text. Even in training stage, things like multimodal inputs and RLVR have joined the fray.

I don't think "create novel concepts" is a real limitation at all. Nothing prevents an AI from inventing new notations. GPT-4o would often do that when talking to AI psychosis victims.

[sublinear]

Language is an interface between whatever our thoughts actually are and the outside world.

Imagine trying to write apps without thinking about the limitations of the APIs you use. In fact we just recently escaped that same stupidity in the SaaS era! That's how silly LLMs will seem in the near future. They will stick around as the smarter chatbots we've wanted for so long, but they are so very far away from AGI.

[ACCount37]

And? Even if I believed this to be a limitation, I could bolt an adapter to an LLM to make it input and output non-text data.

That's how a lot of bleeding edge multimodals work already. They can take and emit images, sound, actions and more.