[Jack000]

LLM failure modes are caused by the lack of external context - they perform poorly on visual tasks because they have no sense of vision for example. Hallucinations are another aspect of this - as embodied agents, humans and animals have a strong bias for counterfactual reasoning because it is needed to survive in a complex information-rich environment (if you believe in something that is false, you tend to get eaten)

The real solution to these problems is to train transformers on a more human-like information context rather than pure text. Hallucinations should naturally decrease as LLMs become more "agentic"

[nomel]

In my mind, I have a "confidence" in my memories and what I know, which seems to be based on how much "context" I can tie it to. This is how I can identify false memories, and say "I don't know".

Is there some "confidence" coefficient that we can extract from AI?

I would claim that hallucinations are required for creativity and problem solving. A "novel" answer is a hallucination to the existing dataset. For a simple example, have ChatGPT-4 come up with a new words that combine two concepts. I imagine this wouldn't be possible if hallucinations weren't allowed.