|
|
|
|
|
|
by Leptonmaniac
21 days ago
|
|
|
I am apparently someone who did not quite catch up to the AI lingo, so parts of the explanation are confusing to me (what is a "vault", why does this thing do stuff in a "loop", how is this more "agentic" than a cronjob). Is my understanding correct that a normal LLM is stateless in the sense that when you talk to it today about frying pans, it does not remember that you spoke about fried rice yesterday? Is this solution effectively adding Markdown files as part of the prompt? Essentially writing into a file "whenever I talk about scripting, I explicitly mean the zsh"? |
|
|
When you chat with an LLM, there's a concept called 'conext'. In essence, context is feeding all previous messages into the LLM together with your latest message. Because context is essentially a finite resource (it requires system memory and increases processing time) the bigger AI providers use tricks to compress context.
These providers usually also have 'memory', which in essence is just parts of previous chats that are entered into the current context based on their relevance. I don't know exactly how this works, but I'd imagine that it does some search for related chats and then adds summaries of those.
In essence, this tool allows you to do those things locally. This allows you much more control of what history the LLM gets and therefore the 'context' it works with. This is important, because context can get dirty. You can notice this if you're chatting with an LLM, it goes completely the wrong way, you try to get it on the right track again, but it just won't. That's because it just tries to predict the next word based on the full context and it might end up consistently predicting the wrong next word.