[gabriel666smith]

The whole thing was borne out of wanting to keep costs low! My favoured approach (last time I was doing this) is using only a tiny sliding context window based on message pairs, rather than tokens, and only for the agents that need it. Amending prose style, for example, shouldn't need context beyond the message it's working on, and then its system prompt.

For the models that require context, I personally found combining a tiny sliding window with a lazy version of the "Recursive Language Models" approach broke immersion least often and had a significantly lower cost. That + the "Id noise" + the strict agents also allowed cheaper models to overperform for me personally.

My lazy version of the RLM approach is basically just giving the agent a grep tool across the full message history & "lore" documentation created by agents, combined with repeated, low-context turns, and a "submit answer" tool for when it felt like it had finished working.

When I looked at the internals of what each agent turn looked like, it did look like a complete mess - but the context window only needs to surface the things it actually needs to know each turn.

Short outputs help a lot with immersion, too - brevity means there is a lot less you can get wrong, and also aids response time & cost.

It does take me an awful lot of prompt tuning to get what I want creatively from LLMs in any format, especially weaker models working in this chain, but I think that's likely always going to be true. Art can have rules, but that doesn't make it science :-)

The RLM approach is detailed here, and I've found it really useful for any cost-sensitive/long-context task: https://alexzhang13.github.io/blog/2025/rlm/

[pitched]

Is this lazy version of RLM where we use subagents and get them to output markdown documents? With one root agent coordinating, it can create a reasonable prompt and provide paths to the markdown documents for the next agent. This keeps context really low because each subagent is filtering the context needed for the next by bouncing it through that document.

Proper RLM looks like you’re allowing agents to directly modify their own context though, like closing browser tabs they don’t need anymore. I haven’t seen anyone actually doing this though.

[gabriel666smith]

When I'm doing it, it depends on whether the agent has ownership of an MD (or other) doc in the flow. If they do, this remains either in-context or greppable, each fresh-context turn.

If not, my agent-level chains just look like:

'''

Turn 1: OK, my task is X, so I should grep for it. Oh, it produced these results:

(Message pairs)

I should expand the context around those message pairs that look relevant.

(3 message pairs around search result)

I should save 1 of these, as it contains relevant information.

[Enforce Tool call limit]

[Delete all context added, except the search tool used already, and the relevant result(s) found.]

Turn 2: OK, my task is this, and it seems I already have this result, but I still need...

...

Turn 10: OK, after that search, my answer is:

[Response]

'''

I've never bothered to let agents self-remove from context, so I would guess it's 'lazy' in that sense. It seems more complex than the task requires in this case, though I can see the benefits on more complex tasks. If you're already saying "this is relevant info", I figure it's simplest to just enforce deletion of everything not marked relevant. In chained prompts, when you're trying to keep costs low and use weaker models, it seems best to limit decision-making as much as possible to make the models as deterministic as possible (on really dumb tasks like, "What is the colour of this goblin's hair?").

There are likely other parts of the actual paper's implementation where the ways I'm implementing it are lazy (because I'm doing this stuff for artistic/fucking around reasons, rather than to advance the field, or implement perfectly), and I think there are various interpretations of what "RLM" should mean. But I found the original paper very helpful, with lots of interesting ideas in, and think it's one where people can take what they need from.

[maxignol]

Well, I had not heard of RLM before, just read the paper, thank you for introducing me to your lazy version !