[simonw]

Don't forget the importance of data privacy. Updating a model with fresh information makes that information available to ALL users of that model. This often isn't what you want - you can run RAG against a user's private email to answer just their queries, without making that email "baked in" to the model.

[viraptor]

You don't need to update the whole model for everyone. Fine tuning exists and is even available as a service in openai. The updates are only visible in the specific models you see.

[simonw]

Maintaining a fine-tuned model for every one of your users - even with techniques like LoRA - sounds complicated and expensive to me!

[lowdest]

It is, but it's also not that bad. A copy of the weights is X GB of cloud storage, which can be stored as a diff if it helps, and added compute time for loading a custom model and unloading for the next customer. It's not free, but it's an approachable cost for a premium service.