[disgruntledphd2]

Nope, those are very different. Fundamentally the reason that LLMs hallucinate is because they are predicting the next most likely word. It would be expensive to find a way to remove outputs where P truth is low, but it's not completely implausible like removing software bugs.

[kevinventullo]

I suspect removing outputs where P(truth) is low is about as hard (and arguably a superset of) detecting code for which P(bug) is high.

As an aside, I think these are both roughly equivalent to detecting code for which P(halts) is high!