[lyaa]

Well, yeah, these models can not interact and observe the world to test the veracity of claims. They are language models and the target for them is text production. No one expects them to understand the universe.

My comment was in response to > GPT-3 is probably a better approach to knowledge processing

and the paper is relevant in that it shows the limitation of current language models in terms of logical consistency or measures of the quality of text sources. GPT-3 and other models are not trained for this and obviously they fail at the task. This is evidence against them being a "better approach to knowledge processing."

Even if we trained future models preferentially on the latest and most cited scientific papers, we will still have issues with conflicting claims and incorrect/fabricated results.

However, that does not mean that it would not be practically useful to figure out a way to include some checks or confidence estimates of truthfulness of model training data and responses. Perhaps just training the models to answer that they don't know when the training data is too conflicted would be useful enough.