[Workaccount2]

People aren't claiming that they are holding textbooks in their model, that would just be even more evidence of reasoning (the LLM would have to reason what textbook to reference, and then extrapolate from the textbook(s) how to solve the problem at hand - pretty much what students in school do; study the textbook and reason from it to answer new test questions)

People are claiming that the models sit on a vast archive of every answer to every question. i.e. when you ask it 92384 x 333243 = ?, the model is just pulling from where it has seen that before. Anything else would necessitate some level of reasoning.

Also in my own experience, people are stunned when they learn that the models are not exabytes in size.

[sigotirandolas]

I think the more realistic argument is that the model can generalize, but only by learning shortcuts (e.g. how to pattern match a problem to a likely answer) and simple algorithms (e.g. how to propagate carries in a multiplication). And this only appears intelligent because this pattern matching is really good and backed by a huge amount of compressed/memorized answers.

The AI pessimist's argument is that there's a huge gap between the compute required for this pattern matching, and the compute required for human level reasoning, so AGI isn't coming anytime soon.

[RugnirViking]

I think I hadn't seen the ai pessimist side laid out like that before,but i feel like i understand it a lot better now. Definitely worth a good think at least. The difference between reproducing an algorithm you've seen before Vs ongoing following the steps required and course correcting, showing understanding. I feel like this can be seen in other high profile failures both in AIs and in humans.

[TeMPOraL]

> I think the more realistic argument is that the model can generalize, but only by learning shortcuts (e.g. how to pattern match a problem to a likely answer) and simple algorithms (e.g. how to propagate carries in a multiplication).

This is exactly what humans do too. Anything more and we need to use tools to externalize state and algorithms. Pen and paper are tools too.

[sigotirandolas]

My thought is that we humans are bad (by computer standards) at arithmetic and memorization because those are not evolutionarily useful on their own.

On the other hand general problem solving is, and so far any attempt to replicate it using computer algorithms has more or less failed. So it must be more complex than just some simple heuristics.

Perhaps the answer is just "more compute" but the argument that "because LLMs somewhat resemble human reasoning, we must be really close!" (instead of 25+ years away) seems wishful thinking, when:

(1) LLMs leverage a much bigger knowledge base than any human can memorize, yet

(2) LLMs fail spectacularly at certain problems and behaviours humans find easy

[thomasahle]

> On the other hand general problem solving is, and so far any attempt to replicate it using computer algorithms has more or less failed.

Well, this is what the whole debate is about isn't it? Can LRMs do "general problem solving"? Can humans? What exactly does it mean?

[sigotirandolas]

A lot of it is being able to make reasonable decisions under novel and incomplete information and being able to reflect and refine on their outcome.

LLMs's huge knowledge base covers for their incapacity to reason under incomplete information, but when you find a gap in their knowledge, they are terrible at recovering from it.