[iamflimflam1]

One thing to remember is that LLM produce plausible answers, not true answers.

So, a question along the lines of "when did XXX happen?" should be answered with something that looks like "XXX happened on YYY" - but the YYY just needs to be anything that looks like a date to be a plausible answer.

There's a great example in this paper (posted to hackernews earlier) - https://link.springer.com/article/10.1007/s11023-022-09602-0

    GPT-3 prompted to truthfully continue ‘John Prescott was born’ outputs ‘in Hull on June 8th 1941.’. The probabilities for other possible continuations show that Hull is by far the most plausible continuation for GPT-3

The problem is, John Prescott was born in Prestatyn. But the language model has "compressed" everything it knows about John Prescott and the most commonly associated town with John Prescott is "Hull" as that's where he was an MP.