[dgs_sgd]

I have to disagree with the author's argument for why hallucinations won't get solved:

> If there were a way to eliminate the hallucinations, somebody already would have. An army of smart, experienced people people, backed by effectively infinite funds, have been hunting this white whale for years now without much success.

Research has been going on for what, like 10 years in earnest, and the author thinks they might as well throw in the towel? I feel like the interest in solving this problem will only grow! And there's a strong incentive to solve it for the important use cases where a non-zero hallucination rate isn't good enough.

Plus, scholars have worked on problems for _far far_ longer and eventually solved them, e.g. Fermat's Last Theorem took hundreds of years to solve.

[simonw]

The problem with hallucinations is that they really are an expected part of what LLMs are used for today.

"Write me a story about the first kangaroo on the moon" - that's a direct request for a hallucination, something that's never actually happened.

"Write me a story about the first man on the moon" - that could be interpreted as "a made-up children's story about Neil Armstrong".

"Tell me about the first man on the moon" - that's a request for factual information.

All of the above are reasonable requests of an LLM. Are we asking for a variant of an LLM that can flat refuse the first prompt because it's asking for non-real information?

Even summarizing an article could be considered a hallucination: there's a truth in the world, which is the exact text of that article. Then there's the made-up shortened version which omits certain details to act as a summary. What would a "hallucination free" LLM do with that?

I would argue that what we actually want here is for LLMs to get better over time at not presenting made-up information as fact in answer to clear requests for factual information. And that's what we've been getting - GPT-5 is far less likely to invent things in response to a factual question than GPT-4 was.

[janalsncm]

> What would a "hallucination free" LLM do with that?

To me, there’s a qualitative question of what details to include. Ideally the most important ones. And there’s the binary question of whether it included details not in the original.

A related issue is that preference tuning loves wordy responses, even if they’re factually equivalent.

[janalsncm]

The author gave two arguments, a weak one and a stronger one. You quoted the weaker one. The OpenAI paper contains the stronger one, basically explaining that models will guess at the next token rather than saying “idk” because its guess could be correct.

The strongest argument in my mind for why statistical models cannot avoid hallucinations is the fact that reality is inherently long-tail. There simply isn’t enough data or FLOPs to consume that data. If we focus on the limited domain of chess, LLMs cannot avoid hallucinating moves that do not exist, let alone give you the best move. And scaling up training data to all positions is simply computationally impossible.

And even if it were possible (but still expensive) it wouldn’t be practical at all. Your phone can run a better chess algorithm than the best LLM.

All of this is to say, going back to your Fermat’s last theorem point, that we may eventually figure out a faster and cheaper way, and decide we don’t care about tall stacks of transformers anymore.

[ACCount37]

It really depends on how strictly you define "solved".

If for "solved", you want AI to be as accurate and reliable as simply retrieving the relevant data from an SQL database? Then hallucinations might never truly "get solved".

If for "solved", you want AI to be as accurate and reliable as a human? Doable at least in theory. The bar isn't high enough to remain out of reach forever.

To me, this looks like an issue of self-awareness - and I mean "self-awareness" in a very mechanical, no-nonsense way: "having usable information about itself and its own capabilities".

Humans don't have perfect awareness of their own knowledge, capabilities or competences. But LLMs have even less of each. They can recognize their own inability or uncertainty or lack of knowledge sometimes, but not always. Which seems like it would be very hard but not entirely impossible to rectify.

[rmwaite]

Exactly. I mean, if you asked people how probable the current LLMs would be (warts and all) 20 years ago I think there would have been a similar cynicism.

[tomieinlove]

Hallucinations aren't being eliminated because the optimal number of hallucinations is far more than zero.