[bastawhiz]

That's not the point. LLMs work by predicting what text to generate next. It doesn't work by choosing facts, it works by saying the thing that sounds the most appropriate. That's why it's so confidently wrong. No amount of training will eliminate this problem: it's an issue with the architecture of LLMs today.