[matthewmacleod]

I don’t think it’s splitting hairs, but I also don’t think it’s a criticism to say that a model like this lacks “understanding”. It’s not a bad thing; if anything, I’d say it’s less “respectful” to imbue a digital construct with characteristics it doesn’t have, rather than embracing those which it does!

What I mean by lacking understanding is that ChatGPT responds to most queries the same way I'd expect if I asked a human being without understanding of a topic to write about it. The content it produces feels much like the sort of writing I'd do myself when I end up having to write a few paragraphs about something I don't really know or care about – a kind of veneer of reasonableness, like a short high-school five-paragraph essay.

What I find fascinating when it produces very confident, very incorrect output – that's what highlights a lack of what I'd call "understanding". As an example – last week someone shared with me a regular expression it had generated to match a specific string. It did this correctly, and included a detailed explanation about what it had done that was written in complete, comprehensible sentences but was wildly incorrect, in a sort of self-contradicting way that even the worst human writer would struggle with. It seemed very obvious that it was stringing together words that kind of made sense together, but that there was no abstract model of the thing it was describing.

[JoeAltmaier]

I've noticed the same. It speaks loudly and confidently, with no logic or evidence whatsoever. Just like a blowhard who 'heard something somewhere' and repeats it as fact.