[hatthew]

I think most debates about LLMs understanding boil down to different definitions of the word "understand." For example, with the definition of "understand" that I typically use in my daily life, I would argue that in the chinese room, the system as a whole "understands" chinese.

[cauch]

Fair enough, but then, a pocket calculator also understands math, and a pocket translator also understands language. And a wikipedia page that inform you about radioactivity understands nuclear physics. Some will maybe say it is the case, but if we talk about the LLM capabilities as a novelty, then it implies that we are talking about something else, because otherwise, it is not novel at all and it does not make sense to pretend it is.

[hatthew]

I'd say that broadly speaking, a system understands things when it can interact with them "correctly". I agree a pocket calculator understands math, but I'd say a pocket translator understands grammar, not language as a whole. A wikipedia page does not interact with anything, so I'm not interested in pushing the definition that far. However, if the wikipedia page were to make recommendations for nuclear safety based on some context it receives as input (say via an integrated LLM), I'd be happy to argue that it understands [that part of] nuclear physics.

I don't think that LLMs as black boxes are fundamentally novel, I just think that their internal design is novel, and their generality and ability to give correct responses to complex topics is far beyond anything previously. For example I would argue that wolfram alpha has a poor understanding of language and a very good understanding of math. I would argue that LLMs have an excellent understanding of language and a mediocre understanding of math, but are able to temporarily increase their understanding of math through document retrieval and "thinking" (or whatever you want to call the process of iteratively generating tokens that build on each other to result in a final response).

[cauch]

Well, then you basically agree with Chiang's article. Just that Chiang as a clever usage of the word "understanding" than you (more clever because more nuanced: 1) I doubt that "people on the street" will agree that obviously "brainless" objects, like a pocket calculator or an interactive wikipedia page will understands anything, 2) Chiang is not stumbling on words: he explained his case that makes clear what he means, and it is to the interlocutor to adopt his vocabulary (because it is very legitimate here) rather than start saying "hm, no, I disagree, because for me, 'conscious' means 'print something on the screen', so LLMs are conscious". That is just missing the point)

[wan23]

Does a pocket calculator not understand arithmetic? What part of a fourth grader's understanding is missing?

[cauch]

Not sure what you mean.

I'm happy both ways:

Either you say that a pocket calculator understands arithmetic, and that LLM understand language, which is something trivial. If a pocket calculator understands arithmetic, than previous substitutes to calculators, such as an abacus, do too. In this case, a word dictionary also understand language. And it is basically what Chiang's article says: the LLM don't understand language more than a word dictionary does. If you disagree with Chiang, it looks like you do only because you don't understand what he is saying, or somehow are not mature enough to realise that Chiang may use a different definition of "understanding" than yours in a fully legitimate way, like everyone is always doing when talking about plenty of subject.

Or you pretend that a pocket calculator understanding of arithmetic is somehow different than the one of an abacus or other obviously inanimate object who are obviously not thinking.