[davebren]

> What is the difference between what LLM's do and "true" understanding?

The thing where you can understand the meaning of this sentence without first compiling a statistical representation of a 10 trillion line corpus of training data.

Unless you're an NPC of course.

[smashers1114]

I mean brains get a lot of training data too in order to understand language. I don't think you provided a relevant difference.

Or rather, maybe I don't understand what you mean :)

[davebren]

When you think about the word apple and what it signifies, what do you experience? Is there a feeling of "appleness"? Do you think that sense of meaning is equivalent to the numerical weights of an LLM?

[smashers1114]

When you think about the word apple and what it signifies, what do you experience?

So I have all sorts of associations with "apple" and spent a little time playing with it.

First in a raw physical sense I can imagine an apple in my head, spin it around, imagine its physics with near cylindrical symmetry etc. A red apple is what first pops into my head, although of course I know there are many apple variants and have opinions on their taste etc.

There are many cultural associations I have with apples from Newton to George Washington. The company Apple has its own set of ideas that I interact with when I hear the word.

In other words I can think of various associations I have to the word apple of various strengths. These associations and strengths are functions of my experience encountering the word and actual apples.

Is there a feeling of "appleness"?

I don't really know what this would mean. I would say no, unless it can perhaps be defined what appleness means and feels like. I don't really notice any strong set of emotions or feelings from this thought exercise.

Do you think that sense of meaning is equivalent to the numerical weights of an LLM?

Again I think I would need a definition of "sense of meaning". I don't seem to derive a singular pointlike meaning when contemplating a singular word. I never was contending that human and LLM cognition are exactly equivalent, but I could see these association strengths being represented in LLM weights. I would say then if an LLM has similar association strengths with "apple" then it "understands" apples as well as I do. Of course this is really hard to test, but frontier models could give you all sorts of apple facts and cultural associations and so on. It may slip up and hallucinate, and I'm sure that I also believe at least one false thing about apples.

So what is your brightline between LLM and human understanding in this example? I assume that your line of reasoning would argue that LLMs do not understand apples. Why don't LLMs understand the word "apple?

[davebren]

It sounds like you don't have the subjective experience of meaning that most humans do, so maybe that would explain why you don't think there is anything beyond associations. Maybe this is the core difference that's determining how people see LLMs.

I'm not sure how I would convey what meaning and understanding is to someone if they don't experience them. This is my poor attempt though: There can not just be associations there need to be "things" to associate between. Otherwise you have no ground, it is all map and no territory. Ultimately it would just be meaningless associations between meaningless symbols.