[alkonaut]

So the memory doesn't solve the context problem of e.g. "conversation context"? I.e. the storage isn't modified while the model is used? If I make an app that makes conversation using such a model model, then the storage isn't modified to insert knowledge about what the early parts of the conversation was about, and it's only bringing a database of fixed information into the conversation? (I have a friend who is just like that).

[tveita]

You could update your storage as you go, the indexing doesn't appear to be that expensive.

For many tasks it wouldn't be helpful because the input is small enough to be covered by the context already, and for summarizing and question answering tasks, you want it to repeat information from other documents, but not from earlier in its own output.

It might be interesting for a long-context task like "given the first parts of this book, complete the next chapter".

[TulliusCicero]

I know AI Dungeon and Novel AI both factor in several recent text inputs when generating new text, and also have a memory section where you can add things you want the AI to never 'forget' about the current story.