[IshKebab]

I believe the word comes from when people started using CNN models in reverse, so hallucinate input that never existed. LLM output is produced via a vaguely similar process. https://en.wikipedia.org/wiki/DeepDream

But in any case, they aren't mistakes. LLMs are not trained to produce true output; they are trained to produce likely output. "Likely" happens to overlap with "true" a lot, but not always. If you ask Claude why aeroplanes fly it will still spew some nonsense about curved wings. Very likely output; not really true.

[d4ng]

LLMs are trained on a model of the world. We rely on them to produce output that correlates with our experience of the world. When we talk about truth, which is simply a word in language with various connotations, and which is a word that is very difficult to define objectively, we talk about what correlates (likeliness) with our shared experience of the world. LLMs have an internal model that is derived from experience, and so do humans.

[watwut]

> LLMs are not trained to produce true output; they are trained to produce likely output.

That is not what google search is promoting. They claim to be search. That is not what AI companies are promoting and selling.