[cjf101]

Claim verification is currently an open problem, as OpenAI admits to in their own promotional docs for ChatGPT. Issue number 1 here seems fairly thorny. And the second statement reveals the real problem: Even if the model was only trained on truths, would tuning it not to deviate from truth make it less able to answer questions?

> ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source of truth; (2) training the model to be more cautious causes it to decline questions that it can answer correctly; and (3) supervised training misleads the model because the ideal answer depends on what the model knows, rather than what the human demonstrator knows.

This is definitely not my area of expertise, but intuitively, it looks like increasing the complexity/varying the training techniques can increase the likelihood of correct answers, but I think the need to give the model leeway to let it work means that ultimately, either human or automated fact checking will need to be incorporated when using this kind of model for fact-finding questions.