[gslepak]

That's like saying we should wait for positive proof of AGI from combustion engines. That'll never happen, no matter how much you tweak the engine. It's just not possible.

The negative proof is there in the definition itself. Transformers are not AGI, they're frozen human intelligence of the autocomplete variety. That can never be AGI and anyone who says otherwise doesn't understand transformers or AGI.

[xscott]

This kind of proof isn't really as water tight as you claim. It's a lot like saying state machines are limited to processing regular expressions, and then completely ignoring how easy it is to add a stack or linear memory to a state machine to make it a PDA or Turing machine.

So yes, the LLMs can be trivialized as just randomized autocomplete, but if you add a database or memory to the side very basic MLPs can become a Turing machine. It's going to take a lot more proof to say a Turing machine could never be intelligent. And you can do more than just give the LLM side memory - you can have them invoked recursively, use message passing as coroutines, and so on...

You might be technically correct if you ignore anything other than the very restrictive definitions you're using, but even there I'm not certain. If you had a LLM with a trillion token window, is that good enough to act as a memory? Human brains aren't infinite either.

[liuliu]

Agreed. It is nonsensical to argue that a 3B transformer that hard-capped to decode 100 tokens is "intelligent". Of course when we are evaluating whether "transformers" is intelligent or not, we are talking about taking transformers as a core part of the system in some ways and enhance it with some other means (as you said, it is pretty trivial to making transformers a Turing machine, hence can carry out any compute, including intelligence (if you are in the camp that intelligence is computable, I don't think it makes sense to argue with anyone who otherwise believes intelligence is not computable)).

[xscott]

Lol, I totally agree about anyone using the non-computable angle.

However, I've got a 20GB GGUF file on my disk that can write code better than 99% of the people I ever worked with in the last 25 years, and ravens seem pretty clever with about 2 billion neurons... I have no idea what the lower bound is.

Fun to think about though :-)

[scoopdewoop]

You are super positive that transformers can't become AGI, wow. Care to explain how atoms _can_ become AGI?

[refulgentis]

Oh! would you mind explaining that out a bit? :)

[gslepak]

See the adjacent thread with @altruios.