[grumbel]

And yet if you see some AstroTurf you'd still call it grass. In the end there is no "true understanding", there are just predictions we make about the world. Depending on how deeply you look, they are often incorrect, but also generally good enough.

GPT isn't quite at the good-enough point and being limited to only text, makes it impossible to reason about aspects of the world that are difficult to describe in text or simply weren't in the training data.

And more generally speaking, the claim that LLMs don't understand anything really doesn't hold up given how much they are able to hallucinate. If a LLM truly wouldn't understand anything, it wouldn't be able to generate plausible text, it would either generate nonsense or be limited to whatever was in the training data, but that's not the case. The LLMs can predict past their trained knowledge and predict stuff they haven't seen yet. Those predictions will sometimes turn out wrong, but so will the humans prediction that the AstroTurf is grass when taking a closer look.