[dennisy]

Not sure if there is a great deal of maths to understand. The output of an LLM is stochastic by nature, and will read syntactical perfect, AKA a hallucination.

No real way to mathematically prove this, considering there is also no way to know if the training data also had this “hallucination” inside of it.

[ben_w]

I think mathematical proof is the wrong framework, in the same way that chemistry is the wrong framework for precisely quantifying and explaining how LSD causes humans to hallucinate (you can point to which receptors it binds with, but AFAICT not much more than that).

Investigate it with the tools of psychologically, as suited for use on a new non-human creature we've never encountered before.