[eskibars]

I wasn't, and I'm not sure how you got that out of what I said. I'm not claiming "understanding," "sentience," etc.

I'm claiming there's a great deal of work in the realm of research that I'm excited by: research I expect to be done by humans. I do expect/am supposing that result may be LLMs of a different nature: to apply guard rails around generative LLM systems, but that's not to anthropomorphize them: just to suppose their purpose.

The term "hallucination" does anthropomorphize LLMs, but I think that's now accepted nomenclature in the industry, at least for the time being, and it's helpful to have some standard nomenclature to describe some of the benefits and problems.

[ajcp]

"The term 'hallucination' does anthropomorphize LLMs"

It does not as hallucinations are not only something humans experience. Beyond that it is now an accepted term of art used to describe a specific behavior exhibited by an LLM that is separate from the biological one.

The problem I have with the term is that we already have one that describes much more accurately what these models are doing: it's called guessing. Guessing is simply reporting information one does not know to be true. As a model does not have data points regarding certain information, each token it returns is done so with lower and lower confidence. It's literally guessing. But since we aren't exposed to the confidence score of the completion it's taken to be full confidence, when it is not the case.

[jamilton]

That framing fails to describe the case where the model is confident in a response (at the token-level), and is wrong, which I think is still considered hallucinating.

[ajcp]

How can a model return high-level confidence at a token level on data it can't predict over.

[jamilton]

Misconceptions. There's no inherent reason a false statement would have lower probability than a true one.

To be clear, I'm referring to things like GPT-3.5 reportedly consistently messing up on statements like "what's heavier, two pounds of feathers or a pound of bricks". Being consistently wrong in the same way implies to me (but I don't know for sure) that the class of response is high probability in an absolute sense.

I can't find the article that demonstrated the sort of things that GPT consistently gets wrong, but it was things like common misconceptions and sayings.

[ajcp]

Very interesting. So it could produce, with high confidence, common and real-world guesses found in it's dataset.

So in that case it's not guessing and not wrong; it's indeed producing something that is correct, but still false. Now we're really getting into the weeds here though.