[Karellen]

> Hallucinations are an engineering problem and can be solved.

I'd like a little more background on that claim.

As far as I've been able to tell from my understanding of LLMs, everything they create is a hallucination. It's just a case of "text that could plausibly come next based on the patterns of language they were trained on". When an LLM gets stuff correct, that doesn't make it not a hallucination, it's just that enough correct stuff was in the training data that a fair amount of hallucinations will turn out to be correct. Meanwhile, the LLM has no concept of "true" or "false" or "reality" or "fiction".

There's no meta-cognition. It's just "what word probably comes next?" How is that just "an engineering problem [that] can be solved"?

[ben_w]

I agree it's more than a simple engineering challenge, but I do so because it is not entirely clear if even humans avoid this issue, or even if we merely minimise it.

We're full of seemingly weird cognitive biases: Roll a roulette wheel in front of people before asking them the percentage of African counties are in the UN, their answers correlate with the number on the wheel.

Most of us judge logical strengths of arguments by how believable the conclusion is; by repetition; by rhyme; and worse, knowledge of cognitive biases doesn't help as we tend to use that knowledge to dismiss conclusions we don't like rather than to test our own.

[Mawr]

How is that bias weird? It has a straightforward explanation - the visual system has an effect on reasoning. This, as well as other human biases, can be analyzed to understand their underlying causes, and consequently mitigated. LLM output has not discernible pattern to it, you cannot tell at all whether what it's saying is true or not.

[ben_w]

> How is that bias weird?

The people can see a random number that they know is random, and yet be influenced by it when attempting facts.

> LLM output has not discernible pattern to it, you cannot tell at all whether what it's saying is true or not.

LLMs are the pattern. This is a separate axis to "is it true?"