[zerop]

The explanation of "hallucination" is quite simplified, I am sure there is more there.

If there is one problem I have to pick to to trace in LLMs, I would pick hallucination. More tracing of "how much" or "why" model hallucinated can lead to correct this problem. Given the explanation in this post about hallucination, I think degree of hallucination can be given as part of response to the user?

I am facing this in RAG use case quite - How do I know model is giving right answer or Hallucinating from my RAG sources?

[kittikitti]

I incredibly regret the term "hallucination" when the confusion matrix exists. There's much more nuance when discussing false positives or false negatives. It also opens discussions on how neural networks are trained, with this concept being crucial in loss functions like categorical cross entropy. In addition, the confusion matrix is how professionals like doctors assess their own performance which "hallucination" would be silly to use. I would go as far to say that it's misleading, or a false positive, to call them hallucinations.

If your AI recalls the RAG incorrectly, it's a false positives. If your AI doesn't find the data from the RAG or believes it doesn't exist it's a false negative. Using a term like "hallucination" has no scientific merit.

[esafak]

So you never report or pay heed to the overall accuracy?

[miraculixx]

"Hallucination" is just to term we use to say "this result is not what it should be". The model always uses the very same process, it does not do one thing for "hallucinations" and something else for "correct" results.

In a nutshell it is always predicting the next token from a joint probability distribution. That's it.

All other interpretations are speculative.

[pcrh]

The use of the term "hallucination" for LLMs is very deceptive, as it implies that there is a "mind".

In ordinary terms, "hallucinations" by a machine would simply be described as the machine being useless, or not fit for purpose.

For example, if a simple calculator (or even a person) returned the value "5" for 2+2= , you wouldn't describe it as "hallucinating" the answer....

[astrange]

"Hallucination" happened because we got AI images before AI text, but "confabulation" is a better term.