[jstanley]

I think you may be confused about how LLMs work. Editing the history you send to the API works perfectly fine. You don't need the LLM to specifically "re-evaluate" the history from the point you edited.

In a way it is "re-evaluating" the entire history for every token it generates.

[sureglymop]

What I'm saying is: If, earlier in the history, I change something, that thing may also still be used further down in the history which I specifically don't want.

So yes, I know that I can just do a simple text replacement somewhere in the history but that's not really useful. I want the conversation to be re-evaluated from that point because it might have gone differently with that changed message.

[quantadev]

Yeah, that's where the 'branching' comes in for sure. Ideally a chatbot would be 'tree-based' where you can just go back to a prior point where you wish you had said something different, and just pickup as if you were back at that point in time, while the other parallel branches are ignored.

The way this is done, technically of course, is you build the "Context" by walking back up the tree parent by parent, until you reach the root, building up 'context' as you go along, in reverse order. That will then always be the 'right' context, based on branching.

[haiku2077]

I can do this with my assistants in Zed and Kagi just fine? Did a quick test and it works exactly how you describe.

[sureglymop]

No. When you edit one of your queries in Kagi Assistant, any queries you made in the conversation after that are discarded. Again, I don't just mean simple re-evaluation of a single query.

[haiku2077]

My previous queries aren't discarded. I can flip between them like branches in a conversation tree.

[justinrubek]

Precisely. They are discarded from the new query.