A system that "predicts the next token" in such a way that it is indistinguishable from a human, is just like a human in practice yes.
How does a human decide which word to use in your mind? Magic?
No, it's a logically based biological/neurological process through which at the end of it, you've decided on a word. They are both forms of computing that can produce largely indistinguishable output... doesn't matter that one is biological and the other isn't
It's quite possible a human forms a full representation of a coherent thought before translating that thought into words, meaning no token by token prediction.
Right, but that would still be functionally quite different from what an LLM does, given that it has no full thought and is entirely predictive by token.
And digital audio will never match analog. Can anybody tell the difference anymore?
If a machine produces highly similar output does it matter? LLM's clearly exhibit behavior of having implicitly learned systems, which is the valuable part of intelligence. Humans infer systems through text too, by the way.
How does a human decide which word to use in your mind? Magic?
No, it's a logically based biological/neurological process through which at the end of it, you've decided on a word. They are both forms of computing that can produce largely indistinguishable output... doesn't matter that one is biological and the other isn't