[disgruntledphd2]

> Except when they "extract" something that wasn't in the source. And now what, assuming you can even detect the tainted data at all?

I mean, this is much less common than people make it out to be. Assuming that the context is there it's doable to run a bunch of calls and take the majority vote. It's not trivial but this is definitely doable.

[grey-area]

I really don’t think that’s doable because why do you the majority output is correct? It’s just as likely to be a hallucination.

If he problem is the system has no concept of correctness or world model.

[disgruntledphd2]

Assuming that hallucinationd are relatively random it's true. I do believe that they happen less often when you feed the model decent context though.

[Piskvorrr]

I mean, it is obvious for a human inspecting the one specific input and output sample, but how do you do this at scale? (Spoiler: cross your fingers and hope, that's how)