[FeepingCreature]

> Humans are not isolated nodes, we are more like a swarm, understanding reality via consensus.

> The situation you described is possible, but would require something like a subverting effort of propaganda by the state.

Great! LLMs are fed from the same swarm.

[stoniejohnson]

I was responding to the back and forth of:

> If you pretrained an LLM with data saying Moscow is the capital of Connecticut it would think that is true.

> Well so would a human!

But humans aren't static weights, we update continuously, and we arrive at consensus via communication as we all experience different perspectives. You can fool an entire group through propaganda, but there are boundless historical examples of information making its way in through human communication to overcome said propaganda.

[ben_w]

The main reason for keeping AI static is to allow them to be certified or rolled back (and possibly that the companies can make more money selling fine tuning) — it's not an innate truth of the design or the maths.

[stoniejohnson]

While those are good reasons to keep the weights static from a business perspective, they are not the only reasons, especially when serving SOTA models at the scale of some of the major shops today.

Continual/online learning is still an area of active research.