[int_19h]

The emphasis on "hallucinations" is misplaced from this perspective, IMO. Thing is, when models do hallucinate, they still reason about what they hallucinated. Larger ones (e.g. GPT-4) can even spot their own hallucinations. That is nothing like what we had in the 60s, or even 10 years ago.

[WorldMaker]

I dislike the term "hallucinations" because I feel it also anthropomorphizes the process too much. Unfortunately, "random garbage output" is too many words, but that's closer to what I meant everywhere I used that word.

> Larger ones (e.g. GPT-4) can even spot their own hallucinations.

I've not yet been convinced that this is actually what is happening from the examples I've seen. It all looks to me like more "random garbage output" that "feels correct" but isn't provably correct. Most examples I've seen so far look too much like "Stochastic Crow Mode" [1]. It is prompts and questions that are doing much more work on the humans reading them (and our interests in anthropomorphizing them or mythologizing them) than the LLMs answering them.

[1] https://fediscience.org/@ct_bergstrom/110182336553459017