[jmward01]

So, I've been working on infinite context models (think fixed size state with a few tricks) and I think this will eventually lead to a kind of lock-in by vendor. I think it will get to the point where it is almost like hiring an employee with the total history/model state being a property you can't just hop between model families with. Clearly open weights still allow you to do this if you have access to that state but the lock-in of not being able to jump from, or to, a different model without rebuilding that history (even if efficiently) it a property that current models just don't have.

[CamperBob2]

A related point is that when these things do gain object permanence and the ability to consolidate memories in a more human-like fashion, that's when the vendor lock-in effects will really show themselves.

Those memories will of course reside entirely on the vendor's servers, and there will naturally be no concept of "exporting" them or allowing the user to interact with them directly. At least not at first. Ownership of memories and context will likely end up as subjects of (far) future lawmaking. As if companies like OpenAI and Anthropic didn't already have massive incentives to establish early regulatory capture.

[tclancy]

How are you storing the data that it can't be exported? Or that I can't use an interface or proxy that logs all my conversations locally for import?

[jmward01]

It isn't about storing the data you send/receive. It is about having a model live through millions, or billions, of tokens doing the job. The current state becomes very important. Especially when you think about how interacting with these things works. No system prompts, just keep working with it until it is good at its job and then using that state as a starting point for other jobs. picking a bunch of different states as starting points, etc etc. You didn't use a simple system prompt to get there, you used interaction and examples and just like a person you decided they were good enough at some point and let them start doing the job for real. It is a totally different system than current models use.