[OkayPhysicist]

I played with Eliza before the rise of modern text generation. It's neat, but it's abundantly clear you're talking to a robot. There was no chance that somebody would be fooled into holding a conversation with it. Likewise with Markov-chain based text generators: by the end of a paragraph, it's obviously not human. GPT-3 and beyond absolutely can fool someone who's not aware they're being tested, and frankly in limited conversations it can be difficult even in the non-blind case. That's Turing test passing.

I deliberately avoided the problem solving/doing work side of things in discussing modern AI, because that's a place where there is a significant amount of progress necessary to be useful. I completely agree that it's abilities in that regard, at present, are being grossly overblown. But the ability to parse human language, decipher intent, and synthesize responses in human language very much is a new capability that modern LLMs are extremely good at, and will likely reach reliability levels necessary for autonomous application in the very near future.

A program that can parse human language perfectly absolutely could still hallucinate. When I ask an LLM a question, and it makes up a patently false response, it accurately parsed what I asked of it. It just failed to synthesize a correct response. The parsing of human language and synthesis of information into human language is, in of itself, a powerful capability, that we shouldn't overlook just because it's no longer science fiction.