[aurareturn]

Why not just provide more compute for say, 1 billion token context for each user to mimic continuous learning. Then retrain the model in the background to include learnings.

The user wouldn’t know if the continuous learning came from the context or the model retrained. It wouldn’t matter.

Continuous learning seems to be a compute and engineering problem.

[jacquesm]

Because that re-training is not strong enough to hold, or so it seems. The same dumb factual errors keep coming up on different generations of the same models. I've yet to see proof that something 'stuck' from model to model. They get better in a general sense but not in the specific sense that what was corrected stays put, not from session to session and not from one generation to the next.

My solution is to have this massive 'boot up' prompt but it becomes extremely tedious to maintain.