[llbbdd]

What is the subtle distinction? I'm "many" and it's not clear at all here. If we had some massive physics breakthrough, the LLM needs to be tought about it, but so do people. Teaching people about it would involve producing a host of documents in some format but that's also true of teaching people. Training and learning here seem to be opposite ends of the same verb no matter the medium, but I'm open to being enlightened.

[bjt]

Not sure exactly what the parent comment intended, but it does seem to me that it's harder for an LLM to undergo a paradigm shift than for humans. If some new scientific result disproves something that's been stated in a whole bunch of papers, how does the model know that all those old papers are wrong? Do we withhold all those old papers in the next training run, or apply a super heavy weight somehow to the new one, or just throw them all in the hopper and hope for the best?

[razodactyl]

You approach it from a data-science perspective and ensure more signal in the direction of the new discovery. Eg saturating / fine-tuning with biased data in the new direction.

The "thinking" paradigm might also be a way of combatting this issue, ensuring the model is primed to say "wait a minute" - but this to me is cheating in a way, it's likely that it works because real thought is full of backtracking and recalling or "gut feelings" that something isn't entirely correct.

The models don't "know". They're just more likely to say one thing over another which is closer to recall of information.

These "databases" that talk back are an interesting illusion but the inconsistency is what you seem to be trying to nail here.

They have all the information encoded inside but don't layer that information logically and instead surface it based on "vibes".