[dr_hooo]

On a much simpler level, llm frameworks could re-summarize their context to keep relevant, use-case-specific facts, cleanup and also organize long and short term memory on some local storage, etc. So kind of like sleep. I think these examples are low hanging fruit to improve the perceived intelligence of LLM systems (so probably they're already used somewhere).

[sigmoid10]

We already have that for a while. It works to some degree, but context tokens simply don't offer the level of compression that model weights do. At least with current approaches that keep the context human-readable.