[hatthew]

I'd say that broadly speaking, a system understands things when it can interact with them "correctly". I agree a pocket calculator understands math, but I'd say a pocket translator understands grammar, not language as a whole. A wikipedia page does not interact with anything, so I'm not interested in pushing the definition that far. However, if the wikipedia page were to make recommendations for nuclear safety based on some context it receives as input (say via an integrated LLM), I'd be happy to argue that it understands [that part of] nuclear physics.

I don't think that LLMs as black boxes are fundamentally novel, I just think that their internal design is novel, and their generality and ability to give correct responses to complex topics is far beyond anything previously. For example I would argue that wolfram alpha has a poor understanding of language and a very good understanding of math. I would argue that LLMs have an excellent understanding of language and a mediocre understanding of math, but are able to temporarily increase their understanding of math through document retrieval and "thinking" (or whatever you want to call the process of iteratively generating tokens that build on each other to result in a final response).

[cauch]

Well, then you basically agree with Chiang's article. Just that Chiang as a clever usage of the word "understanding" than you (more clever because more nuanced: 1) I doubt that "people on the street" will agree that obviously "brainless" objects, like a pocket calculator or an interactive wikipedia page will understands anything, 2) Chiang is not stumbling on words: he explained his case that makes clear what he means, and it is to the interlocutor to adopt his vocabulary (because it is very legitimate here) rather than start saying "hm, no, I disagree, because for me, 'conscious' means 'print something on the screen', so LLMs are conscious". That is just missing the point)