[ljosifov]

That's a consideration, for sure. But given the LLM-s have not got the ground truth, everything is controlled hallucination, then - if the LLM tells you an imperfect version of my email or chat, you can never be sure if what the LLM told you is true, or not. So maybe you don't gain that much extra knowledge about me. For example, you can reasonably guess I'm typing this on the computer, and having coffee too. So if you ask the LLM "tell me a trivial story", and LLM comes back with "one morning, LJ was typing HN replies on the computer while having his morning coffee" - did you learn that much new about me, that you didn't know or could guess before?

[int_19h]

There are ways to tell if something is a hallucination or not if you can see all logit probabilities (hallucinations tend to be closer to noise). I'm sure there are other, more advanced statistical analysis methods.