I love this idea! I agree that LLMs often don't have the knowledge that's actually important (a frustration I have with ChatGPT is that it remembers unimportant things and then over-indexes on them in its responses).
Totally agree. The current memory implementations feel a bit meh. you don’t really control what’s stored or how it’s used.
I’ve been thinking of it more as a visual knowledge layer you can inspect + operate on, rather than something implicit inside the model.
Curious if you’ve found any tools that handle this well today?