|
|
|
|
|
by krackers
27 days ago
|
|
>Most agent loops reorder, rewrite, or inject fresh timestamps each turn That's really surprising, since it'd defeat the whole point of KV caching. I mean I buy it considering how sloppily coded the harnesses seem to be, but this like obvious low hanging fruit. I've also often wondered why LLMs aren't trained with a format of having a dedicated contextual system-instruction role at the _end_, which you could use to put context like current time or other misc stuff. |
|
There are context pruning strategies that will prune old messages that are no longer relevant, and context compaction from summaries, etc. But to say "most" do this on "every turn" is overstating things. I think it's more correct to say that "many" do this "occasionally."
I'm also not sure what they mean about injecting fresh timestamps. I could see why you'd prepend/append a timestamp to the user's messages to make the model aware of the current time, and the passage of time, but I can't think of any good reason to edit timestamps in prior messages. I'm sure someone can come up with one, but I'd be very surprised if this was a thing that most agent loops do, let along doing it on every turn.