[vidarh]

I think that if people say LLMs can never be made to think, that is bordering on a religious belief - it'd require humans to exceed the Turing computable (note also that saying they never can is very different from believing current architectures never will - it's entirely reasonable to believe it will take architectural advances to make it practically feasible).

But saying they aren't thinking yet or like humans is entirely uncontroversial.

Even most maximalists would agree at least with the latter, and the former largely depends on definitions.

As someone who uses Claude extensively, I think of it almost as a slightly dumb alien intelligence - it can speak like a human adult, but makes mistakes a human adult generally wouldn't, and that combinstion breaks the heuristics we use to judge competency,and often lead people to overestimate these models.

Claude writes about half of my code now, so I'm overall bullish on LLMs, but it saves me less than half of my time.

The savings improve as I learn how to better judge what it is competent at, and where it merely sounds competent and needs serious guardrails and oversight, but there's certainly a long way to go before it'd make sense to argue they think like humans.

[plaguuuuuu]

Everyone has this impression that our internal monologue is what our brain is doing. It's not. We have all sorts of individual components that exist totally outside the realm of "token generation". E.g. the amygdala does its own thing in handling emotions/fear/survival, fires in response to anything that triggers emotion. We can modulate that with our conscious brain, but not directly - we have to basically hack the amygdala by thinking thoughts that deal with the response (don't worry about the exam, you've studied for it already)

LLMs don't have anything like that. Part of why they aren't great at some aspects of human behaviour. E.g. coding, choosing an appropriate level of abstraction - no fear of things becoming unmaintainable. Their approach is weird when doing agentic coding because they don't feel the fear of having to start over.

Emotions are important.

[vidarh]

Unless we exceed the turing computable - which there isn't the tiniest shred of evidence for -, nothing we do is "outside the realm of 'token generation'". There is no reason why the token stream generated needs to be treated as equivalent to an internal monologue, or need to always be used to produce language at all, and Turing complete systems are computationally equivalent (they can all compute the same set of functions).

> Everyone has this impression that our internal monologue is what our brain is doing.

Not everyone has an internal monologue, so that would be utterly bizarre. Some people might believe this, but it is by no means relevant to Turing equivalence.

> Emotions are important.

Unless we exceed the Turing computable, our experience of emotions would be evidence that any Turing complete system can be made to act as if they experience emotions.

[wat10000]

A token stream is universal, but I don't see any reason to think that a token stream generated by an LLM can ever be universal.

I mean, theoretically in an "infinite tape" model, sure. But we don't even know if it's physically possible. Given that the observable universe is finite and the information capacity of a finite space is also finite, then anything humans can do can theoretically be encoded with a lookup table, but that doesn't mean that human thought can actually be replicated with a lookup table, since the table would be vastly larger than the observable universe can store.

LLMs look like the sort of thing that could replicate human thought in theory (since they are capable of arbitrary computation if you give them access to infinite memory) but not the sort of thing that could do it in a physically possible way.

[vidarh]

Unless humans exceed the Turing computable, the human brain is the existence proof that a sufficiently complex Turing machine can be made to replicate human thought in a compact space.

That encoding a naive/basic UTM in an LLM would potentially be impractical is largely irrelevant in that case, because for any UTM you can "compress" the program by increasing the number of states or symbols, and effectively "embedding" the steps required to implement a more compact representation in the machine itself.

While it is possible using current LLM architectures might make encoding a model that can be efficient enough to be physically practical impossible, there's no reasonable basis for assuming this approach can not translate.

[wat10000]

You seem to be making a giant leap from “human thought can probably be emulated by a Turing machine” to “human thought can probably be emulated by LLMs in the actual physical universe.” The former is obvious, the latter I’m deeply skeptical of.

The machine part of a Turing machine is simple. People manage to build them by accident. Programming language designers come up with a nice-sounding type inference feature and discover that they’ve made their type system Turing-complete. The hard part is the execution speed and the infinite tape.

Ignoring those problems, making AGI with LLMs is easy. You don’t even need something that big. Make a neural network big enough to represent the transition table of a Turing machine with a dozen or so states. Configure it to be a universal machine. Then give it a tape containing a program that emulates the known laws of physics to arbitrary accuracy. Simulate the universe from the Big Bang and find the people who show up about 13 billion years later. If the known laws of physics aren’t accurate enough, compare with real-world data and adjust as needed.

There’s the minor detail that simulating quantum mechanics takes time exponential in the number of particles, and the information needed to represent the entire universe can’t fit into that same universe and still leave room for anything else, but that doesn’t matter when you’re talking Turing machines.

It does matter a great deal when talking about what might lead to actual human-level intelligent machines existing in reality, though.

[vidarh]

I'm not making a leap there at all. Assuming we agree the brain is unlikely to exceed the Turing computable, I explained the stepwise reasoning justifying it: Given Turing equivalence, and given that for each given UTM, there is a bigger UTM that can express programs in the simpler one in less space, and given that the brain is an existence-proof that a sufficiently compact UTM is possible, it is preposterous to think it would be impossible to construct an LLM architecture that allows expressing the same compactly enough. I suspect you assume a very specific architecture for an LLM, rather than consider that LLMs can be implemented in the form of any UTM.

Current architectures may very well not be sufficient, but that is an entirely different issue.