[Eisenstein]

You have described something but you haven't explained why the description of the thing defines its capability. This is a tautology, or possibly a begging of the question, which takes as true the premise of something (that token based language predictors cannot be intelligent) and then uses that premise to prove an unproven point (that language models cannot achieve intelligence).

You did nothing at all to demonstrate why you cannot produce an intelligent system from a next token language based predictor.

What GPT says about this is completely irrelevant.

[coldtea]

>You did nothing at all to demonstrate why you cannot produce an intelligent system from a next token language based predictor

Sorry, but the burden of proof is on your side...

The intelligence is in the corpus the LLM was fed with. Using statistics to pick from it and re-arrange it gives new intelligent results because the information was already produced by intelligent beings.

If somebody gives you an excerpt of a book, it doesn't mean they have the intelligence of the author - even if you have taught them a mechanical statistical method to give back a section matching a query you make.

Kids learn to speak and understand language at 3-4 years old (among tons of other concepts), and can reason by themselves in a few years with less than 1 billionth the input...

>What GPT says about this is completely irrelevant.

On the contrary, it's using its very real intelligence, about to reach singularity any time now, and this is its verdict!

Why would you say it's irrelevant? That would be as if it merely statistically parroted combinations of its training data unconnected to any reasoning (except of that the human creators of the data used to create them) or objective reality...

[Eisenstein]

Let's pretend it is 1940

Person 1: rockets could be a method of putting things into Earth orbit

Person 2: rockets cannot get things into orbit because they use a chemical reaction which causes an equal and opposite force reaction to produce thrust'

Does person 1 have the burden of proof that rockets can be used to put things in orbit? Sure, but that doesn't make the reasoning used by person 2 valid to explain why person 1 is wrong.

BTW thanks for adding an entire chapter to your comment in edit so it looks like I am ignoring most of it. What I replied to was one sentence that said 'the burden of proof is on you'. Though it really doesn't make much difference because you are doing the same thing but more verbose this time.

None of the things you mentioned preclude intelligence. You are telling us again how it operates but not why that operation is restrictive in producing an intelligent output. There is no law that saws that intelligence requires anything but a large amount of data and computation. If you can show why these things are not sufficient, I am eager to read about it. A logical explanation would be great, step by step please, without making any grand unproven assumptions.

In response to the person below... again, whether or not person 1 is right or wrong does not make person 2's argument valid.

[coldtea]

It's not like we discovered hot air ballons, and some people think we'll get to Moon and Mars with them...

> Does person 1 have the burden of proof that rockets can be used to put things in orbit? Sure, but that doesn't make the reasoning used by person 2 valid to explain why person 1 is wrong.

The reasoning by person 2 doesn't matter as much if 1 is making an ubsubstantiated claim to begin with.

>There is no law that saws that intelligence requires anything but a large amount of data and computation. If you can show why these things are not sufficient, I am eager to read about it.

Errors with very simple stuff while getting higher order stuff correct shows that this is not actual intelligence matching the level of performance exhibited, i.e. no understanding.

No person who can solve higher level math (like an LLM answering college or math olympiad questions) is confused by the kind of simple math blind spots that confuse LLMs.

A person understanding higher level math, would never (and even less so, consistently) fail a problem like:

"Oliver picks 44 kiwis on Friday. Then he picks 58 kiwis on Saturday. On Sunday, he picks double the number of kiwis he did on Friday, but five of them were a bit smaller than average. How many kiwis does Oliver have?"

https://arxiv.org/pdf/2410.05229

(of course with these problems exposed, they'll probably "learn" to overfit it)

[Eisenstein]

> The reasoning by person 2 doesn't matter as much if 1 is making an ubsubstantiated claim to begin with.

But it doesn't make person 2's argument valid.

Everyone here is looking at the argument by person 1 and saying 'I don't agree with that, so person 2 is right!'.

That isn't how it works... person 2 has to either shut up and let person 1 be wrong in a way that is wrong, but not for the reasons they think, or they need to examine their assumptions and come up with a different reason.

No one is helped by turning critical thinking into team sports where the only thing that matters is that your side wins.

[ViewTrick1002]

The delta-V for orbit is a precisely defined point. How you get there is not.

What is the defined point for reaching AGI?

[Eisenstein]

I can check but I am pretty sure that using a different argument to try and prove something is wrong will not make another person's invalid argument correct.

[Jensson]

Person 3: Since we can leave earths orbit, we can reach faster than light speed, look at this graph over our progress making faster rockets we will for sure reach there in a few years!

[Eisenstein]

So there is a theoretical framework which can be tested against to achieve AGI and according to that framework it is either not possible or extremely unlikely because of physical laws?

Can you share that? It sounds groundbreaking!

[Terr_]

The people who claim we'll have sentient AI soon are the ones making the extraordinary claims. Let them furnish the extraordinary evidence.

[Eisenstein]

So, I think people in this thread, including me, have been talking past each other a bit. I do not claim that sentient AI will emerge. I am arguing that the person who is saying that it can't happen for a specific reason is not considering that the reason they are stating implicitly is that nothing can be greater than the sum of its parts.

Describing how an LLM operates and how it was trained does not preclude the LLM from ever being intelligent, and it almost certainly will not become intelligent, but you cannot say that it didn't for the reasons the person I am arguing with is saying, which is that intelligence can not come from something that works statistically on a large corpus of data written by people.

A thing can be more than the sum of its parts. You can take the English alphabet, which is 26 letters, and arrange those letters along with some punctuation to make an original novel. If you don't agree that means that you can get something greater than what defines it components, then you would have to agree that there are no original novels because they are composed of letters which were already defined.

So in that way, the model is not unable to think because it is composed of thoughts already written. That is not the limiting factor.

[Terr_]

> If somebody gives you an excerpt of a book, it doesn't mean they have the intelligence of the author

A closely related rant of my own: The fictional character we humans infer from text is not the author-machine generating that text, not even if they happen to share the same name. Assuming that the author-machine is already conscious and choosing to insert itself is begging the question.