[notahacker]

I think the point is the LLM arrives at an obvious and undeniable fact, a misconception common in human discourse, an assertion of something unknowable, a statement of what appears to be opinion, a "creative" response to a brief (both impressive and unimpressive), a reasonable "guess" and a random answer which is only very loosely linked to the prompt in essentially the same way. Humans generally arrive at such responses in different ways, and are often conscious when they're certain, reasonably confident, guessing, needimg an answer to be a particular way to fit their wider goal or bullshitting

So if it's "hallucinating" a probable continuation which asserts something which is [incidentally] understood to be completely wrong or not in the source material by humans, it's going through exactly the same process to arrive at a continuation which [incidentally] is understood to contain only accurate statements or valid summarizations

[hallqv]

If I had a penny each time a human has confidently concluded something that is entirely incorrect… I’ve inadvertently done it countless times myself, and so has every person I know of.

[notahacker]

Oh, everyone's done it, but humans have a lot more ways to consciously or unconsciously reach an entirely incorrect conclusion, and more ways of testing it too.