[cauch]

> I am more, in a sense, arguing that "understanding" isn't clearly defined and sceptical about your confidence that this is an obvious quality of humans.

I did not do that, because it is not what I believe. I don't have any problems with the concept of having non-human being intelligent. It is just that LLMs are not that.

> I'm not getting what this "understanding" thing is in humans that you are talking about

Then that's fine. Other people have a better understanding of the notion. Just simply avoid the conversation if you don't know, it just feels like you are muddying it by not getting the concepts.

> If I re-read your post and replace the word "understanding" with "consciousness" ...

No, I'm not talking about "consciousness". I'm talking about "perceiving the underlying meaning or concept". LLMs don't create their answers by relying on the abstract concepts of the objects they are using, they just have meaningless rules linking the different objects, without grasping the abstract concepts explaining these links.

> I'd definitely pick the "understanding" of Fable over a junior engineer's "understanding" any day

Similarly, I trust better my pocket calculator than a human, but it does not mean that the calculator "understand math", it just has the "math rules" hardcoded without grasping the abstract concepts. In LLMs, the rules are not hardcoded, just extracted from the data, but the LLM doesn't understand any more than a pocket calculator understand math.

[dagss]

Thank you that was clearer.

So, I have a PhD in Astrophysics so I am not a total stranger to doing some thinking. And I would say "create (...) answers by relying on the abstract concepts of the objects they are using" is a lofty goal for humans, something to aspire to more than something that typically goes on. We go by habits and intuition and allegories and quite muddy concepts most of the time. Concepts are malleable and evolve in clarity. And in creating new mathematics etc., intuition, inspiration, "flashes of insights" etc after absorbing oneself in the problem has an important role.

Are these things we have in our minds, whether concepts or habits or intuitions or flashes of insights, better or worse than whatever patterns could potentially be found in the LLM weights?

I struggle to label one of them "understanding" and the other not, at least without involving consciousness somehow.

Obviously you can define "understand" as "understand as a human would" but that is circular and uninteresting.

We just have to agree to find each others position incredible I am afraid :)

[cauch]

> ... is a lofty goal for humans, ... We go by habits and intuition and allegories and quite muddy concepts most of the time

Those are already concepts. For LLMs, the word X is just an object linked to the words W, Y, Z, with no meaning to it. Habits, intuition and allegories are using objects to which we are attributing meaning.

Just to clarify, the links that LLMs create are complex, for example depend on all the surrounding other words, but they are still meaningless. If 2 totally different semantic sets of words happen to have exactly the same graphs of links, then you can swap the sets of word together, it does not matter for the LLM. To use a simplified example where you reduce a set to just 2 words, if "garden" and "pea" are linked the same way that "quantum" and "mechanic", then the relationship are the same for the LLM, without the LLM understanding that "garden pea" is a different concept than "quantum mechanic".

That's what I mean by "understanding": humans understand "garden pea" and "quantum mechanic" as concepts (even if they don't know biology or physics enough to even explain how they work), LLMs just use these objects as meaningless entities. All there is is a graph of links used to generate a sentence, but without knowing what the sentence means. A bit like if someone was giving you all the words of a language you don't know and the exact rules of how to build an answer given an input, but that you don't know what each word means.

Of course, the relationship learnt by the LLMs are very complex, allowing big changes based on the surrounding other 100'000 words. But learning this relationship is still more straightforward than to leap into a conceptual world model (especially because there is nothing to guide the LLM. If "garden pea" and "quantum mechanic" have the same graph geometry, then the world model where "garden pea" is an abstract field of study and "quantum mechanic" is a material object is as probable than the opposite).