[tekacs]

I know a lot of people like to say that compaction makes this moot, but the level of detail you lose across compaction is wildly too much for most things that I do, unfortunately.

Perhaps if your plans don't have as much detail, or if you're not, for example, having a discussion with a lot of nitty-gritty then it's fine?

The lack of long context is the main reason that I still end up using Anthropic.

The worst is when you need it to hold for example a number of papers in its head, or large and complex materials that it needs full resolution on and your context window ends up being perennially at 16%. You have about five minutes of conversation and it compacts and then you have to wait for it to read that again, get to 16%... and repeat.

372 was not perfect, but it was so much better and a godsend. It turned that 12 to 20% into more like 40%.

[jubilanti]

The fact there is no way to disable auto-compaction and no way to go back in the conversation history to before a compact makes codex a no-go for me on any codebase > 5kloc. Even worse it seems to fire randomly at 10-20% of context left. So really you have 80% of 272k as real usable context. Compaction kills my sessions, it hallucinates and is worse than starting fresh. I've had enough times screaming at my computer when it burns tokens on a large codebase, gets to 15%, auto-compacts, and hallucinates so bad it has to read the entire codebase agin, gets to 15%, auto-compacts....

[volker48]

Have you tried using another harness like Pi? You can easily turn off auto compaction and if you leave it on you can use /tree to go back before the compaction if you like.

[sdesol]

> you can use /tree to go back before the compaction

This is really a killer feature in my opinion. I'm currently working on pi brains extension that is designed to solve the compaction issue I have, which is, I don't know what the AI knows after compaction.

https://gitsense.com/screenshots/pi-inspect-overview.png

The screenshot shot above contains what I call a compaction capsule which contains all the messages/events at the point of compaction. The "Files" section in the right side panel shows me how many files were read/modified since compaction.

After compacting, I can easily tell what Pi use to know and use `/tree` to go back in time.

I still wouldn't compact more than 5 times before starting a new session since compaction messages persists in Pi.

[maherbeg]

You also might like https://github.com/nicobailon/pi-boomerang that lets you effectively do some work, compact that section with a summary and continue on.

[behnamoh]

> The fact there is no way to disable auto-compaction and no way to go back in the conversation history to before a compact makes codex a no-go for me

Same. I've told @tibo many times about these issues but apparently OpenAI's focus is on features that buy them the most users ASAP instead of increasing harness quality.

[mycall]

So the practical workaround is to set model_auto_compact_token_limit very near, but still below, the model’s context window. This postpones compaction rather than disabling it.

config.toml

  model_context_window = 27000
  model_auto_compact_token_limit = 17000
  model_auto_compact_token_limit_scope = "total"

[skybrian]

That seems quite different from my design process. I write a plan.md that goes through multiple revisions. The plan is the memory. Restarting and reading the plan again to do another review is a good way to get a different perspective.

[jmalicki]

I like to actually have hierarchical plans.

Start with the big idea. Pin down the product manager-level description, and maybe some details about which features are in or out, how the phased design/roadmap will work, maybe what the basic tech stack will be.

That document gets saved, and has the big picture context.

Next, we go a little deeper, flesh out schemas, APIs, a little more about what the code should look like, the overall testing strategy. This document gets saved, and the AI can reference it for context.

The next level is a very specific implementation plan. Think individual small JIRA tickets in an epic. The AI writes this based on previous context. I make sure each individual step has very explicit instructions about branch names, which branch is branched from where, the testing and documentation that will be done for that step. It should be obvious from the higher level documents, but that helps keep the AI to only worrying about the local document in its context window.

Then have a DAG of stacked pull requests designed for subagents to work through.

All of this keeps the context window down (as well as costs down, and accuracy up) and works super well, and I can get a tremendous amount of code that is easy to review through stacked PRs. Invariably I will want to fix earlier ones then percolate that down the stack, so I have some skills for that.

[spike021]

I try to do similar. I'll even break down parts of the plan into separate docs and have it solely refer to them one by one as needed with /clear between. It's a more manual process but not the worst. I could probably even whip up some kind of skill to automate it somewhat with subagents or similar.

[jmalicki]

You can just tell your main agents "I have these implementation docs in <DIRECTORY>, please spawn parallel subagents to implement all of them" and I generally find that works, and what's great is it's in parallel too so takes less wall clock time (even if there is slight token duplication from having the main agent and the subagent both read the prompt/doc, but overall I think it is less usage of tokens unless you go crazy with /clear).

[Sammi]

So far I have three different folders in my project for three different types of md documents that I keep creating and revising together with the agent: reports, plans, and reviews

Every large feature has at least one of each file created for it in that order. I start by analysing stuff and making reports that might also contain design thinking and decisions. Once I'm satisfied we have properly analysed and thought though whatever it is we're working on then I might make a plan or just jump into implementation depending on how complex the issue is. And then I might have zero or multiple reviews of different kinds by different agents depending on how much verification I think the issue warrants.

This is my super simple worse-is-better development process. I make up what is needed on the spot.

[tekacs]

I have these plan files as well, but it depends on the scope and scale of the things you're executing on, I think. However much detail gets put into the plan, it still doesn't help if part of what the model needs to understand is the fine-grained / perfect detail of a large surface area.

[skybrian]

That sounds more monolithic than modular, but I guess some projects don’t decompose easily? Or maybe they do, but getting there is the goal.

[BoorishBears]

I've found if you pay close attention, models in Codex get briefly lost on where it is in the plan post-compaction. If it was in the middle of a test it often tries to resume the test from the start, or will seem "surprised" that past steps are completed already.

It's hard to believe that sort of confusion doesn't hurt performance a bit, the question is if that degradation is worse than the performance fall off from long-context (which is highly task specific)

[dnautics]

> I write a plan.md that goes through multiple revisions

instrumentally, isn't this the same as just doing plan mode?

Plan mode is just a .md file in a 'well-known' location that is privileged to a certain spot in the kv cache (after the system prompt).

[planckscnst]

I agree. Compaction sucks, so I made tools that let the LLM selectively delete (and recall if needed) chunks of its context. You might want to try context bonsai if you're routinely hitting the auto-compaction wall.

https://github.com/Vibecodelicious/context-bonsai-agents

[tekacs]

I've implemented a similar approach – although I'm surprised not to see mention of cache prefix busting in there!

[planckscnst]

I did calculations on the prefix cache effect on costs of sessions where I used it and found that the removal of tokens from context had a much bigger effect on reducing costs than cache busting had on increasing them. I should re-do that and publish it.

[jeremyjh]

Yes the idea is cool but this could really hammer usage, especially just leaving it up to the agent to decide when to do it. I'm not surprised though, considering the github account is named "Vibecodelicious" and became active in December.

Poking around in the repo the whole implementation is an unsupervised LLM fever-dream.

[planckscnst]

It depends on what you mean by "unsupervised" - I've been strictly working through the agent, through specs the entire time, but it's been very supervised, I just leave the mistakes in-place and have it work from there.

However at this point it can completely maintain itself. When a new version of Claude or OpenCode is released, it updates itself to work on the latest version. It can also add new implementations for harnesses pretty reliably. It's actually pretty fun to watch it at this point. "Make this work on Hermes agent and message me when you're done" and an hour later or so, I can go play with it in Hermes.

[adyavanapalli]

This is actually brilliant. I've seen other approaches where the context is selectively pruned, but never in a recoverable way.

P.S. Love the "bonsai" name btw!

[8note]

ive built a variation that spins the chunks out as if they had been done by a subagent, and so the summary is like what the subagent returns

i couldnt figure out a nice enough way to reload the conversation, and didnt want to have to worry about the cache rebuilds so its a bit languishing

[znnajdla]

I have a hunch that OpenAIs proprietary compaction endpoint actually does something similar on the backend.

[tekacs]

When I've asked Codex agents about things that were in their context window, they've never – to my experience – been able to actually retrieve something from before compaction when using the proprietary compaction endpoint. Instead, they've had to consult their actual transcript.

So... at least as of a week ago or so, I don't believe so.

[jmalicki]

Instead of compaction, I very aggressively do everything with teams of subagents. The main agent keeps the high level context, and all of the nitty gritty detail and code details not needed by the main agent never even make it into the context.

[thejazzman]

This used to be really effective and cheap, now it lights my quota on fire, even after I fix the config and restore smaller subagents instead of like-agents

Multiple (slow) conversations seems to be the efficient path for me the past week or so

[Supermancho]

That is how the majority of developers use AI. Aim small miss small and all that. What's great about AI is that you can ask many many small questions about big data dumps (like logs and stack traces), for relatively little usage.

[ljm]

How does that work out in terms of usage? I imagine that, unless you're on the highest tier subscription or have a no expense spared approach to agentic coding, then you're going to hit a wall pretty fast.

[jmalicki]

It's actually less token usage overall, since the gigantic context that builds up in the main agent is a huge driver of costs.

[cortesoft]

The solution isn’t to stuff everything it needs to know into the context, it is to have it write out what it needs to know for every part. It should write out plans, infrastructure decisions, etc, and then use those in future work as needed.

[HDThoreaun]

Yea by the time you get to 300k context its IQ is a solid standard deviation lower anyway, why would you really want to beyond that?

[CamperBob2]

Exactly. Just about every prompt I issue these days generates or edits an .md file.

[ethin]

I suffered this same problem with Anthropic, ironically. Mainly with reverse engineering things. That would usually end up requiring a ton of context it had to remember, and then compaction would kick in and it would just forget half of it (that half being really important too), and even switching to Claude Code and having it document everything wouldn't really fix the problem. It was honestly quite sad because I've usually gotten better results with it, but it falls a part the moment you try doing something like that that's really complicated and requires a ton of different attempts to get it right.

[eagerpace]

The best way around this for me is planning a project well ahead of time and using subagents to execute individual tasks while the primary agent acts as a PM.

[tekacs]

That isn't really a way around it for the kinds of things I'm talking about. The whole point is that getting all of the relevant context to even orchestrate it takes up most of the model's 272K.

[jmalicki]

For those kind of tasks, I do hierarchical planning, and then parts of the plan can focus one agent on keeping a smaller amount of pre-determined context when orchestrating.

Think: A VP of engineering has some super high level goal: "build a new product to beat our competitor in this new market". It then filters down and various levels of reports build more and more detailed yet more narrow pieces of planning/orchestration. They don't make it work by keeping all of the context in their head and micromanaging.

[eagerpace]

Yes, sometimes planning spans sessions too but I have even that broken down into stages where we start with the architecture and research, then save that, then define the milestones, then break down into the individual issues that the less sophisticated models can implement in a single session.

[whearyou]

I haven’t found I need to plan way too far ahead of time as long as I’ve defined the interface between the tasks

[yearolinuxdsktp]

If you’re using the pi.dev harness, I found this tool to be quite good as a compaction alternative: https://pi.dev/packages/pi-blackhole — it keeps a memory so every prior conversation piece is recallable, not lost.

[voxgen]

Pay attention to what it forgets, and start telling it to proactively note down those things into nominated files while it works, e.g. indexing topics covered by each chapter/section/page of each paper/material. GPT-5.6 is really good at finding context again, if given the smallest hint where to start looking.

You can also ask it to read your past sessions, find places where it was wrong after compaction, and figure out a strategy to persist the most important summary information.

I have several Hermes threads that have each had >10 compactions with a 200k context limit, and with the right instructions on note-taking, they require at most a "double-check that against our past decision records" to be put back on track after a slip-up.

[reinitctxoffset]

You can mad customize it if you're willing to set up like, oh my pi or whatever.

I didn't quite get it tuned up enough to be a daily driver but I'm basically sure it can be hotrodded however you want. It comes out of the box with like four different vendor hostile compact strategies, including compacting into images at the smallest size Claude can read, which the "coding is basically solved" geniuses conveniently leak the resolution heuristic out of their website along with all the other side channels they print for the MSS.

[sidewndr46]

I think I've seen the opposite end of this as well: I get the most value from any LLM when I ask it one very targeted question with a followup. I don't fall outside of any sort of context window sizing when I am doing that. That isn't to say that I don't have a few long engineering discussions that are helpful, but most are very to the point. It helps that I have decades of experience at this point and effectively know what to ask to narrow the scope quickly.

[rcarmo]

Depends on what you are using. I have been experimenting with various compaction techniques in piclaw (https://github.com/rcarmo/piclaw/blob/main/docs/pipelined-co...) and Codex has native "opaque" server-side compaction at a dedicated OpenAI endpoint.

[PeterStuer]

All depends on context coherence. E.g. in Claude Code with the "1M context" models I am reluctant to push past 30%. So if that Codex 272K is 100% coherent than it is less of a difference. Still, even then the "hard fail" boundary has moved. Longer running agentic processes will surely run into this more often.

[keeganpoppen]

sometimes compaction is a death knell and other times it's totally fine. it really just depends on whether the thing you were doing at the very end has high correlation with stuff from early on in the session, or if there was an obvious ~break s.t. you were just on some specific subtask. the situation has clearly gotten a lot better than even 6 months ago-- i think openai handles compaction better in my own experience. but it's exactly the kind of thing that yegge was trying to get at with gastown-- i used to run every session s.t. i could re-animate it every time the compaction boundary came... now, when the compaction boundary comes i am so pot committed that i just hold my breath.

[azuanrb]

Curious, what kinds of tasks do you do that require such a large context window? Anything specific?

[serf]

for me it's often codebase decomposition.

it's hard to modularize a monolith without keeping large chunks in context at first pre-delegation; the orchestrator -- however you implement it -- needs to carry as much of the original thing as possible into context and big monoliths make this task heavier.

so, tl;dr : I use a lot of tokens re-writing legacy codebases that were constructed with very little CS training in some huge spaghetti fashion by random people around the world.

[QuantumGood]

Compaction is a feature like swapping out a car for a tricycle is a feature for greater economy.

[matheusmoreira]

What. Anthropic's models have million token context windows! I'm switching to OpenAI next month, I can't believe they're still stuck on ~300k... Better start adapting to the new reality I guess.

[scotty79]

OpenAI compaction is very different than Anthropic compaction, especially in 5.6

I use GLM5.2 and gpt-5.5 and 5.6 and never noticed any quality degradation near the limits of the context or due to compaction.

[serf]

>I use GLM5.2 and gpt-5.5 and 5.6 and never noticed any quality degradation near the limits of the context or due to compaction.

that's because you and your workloads probably fall into a bucket of users where compaction is tuned well.

but the reality is that those processes need to pick what to carry over, and that's a tough thing to get right for everyone, so for me compaction is a signal to restart the session or to tell the LLM to reference existing design docs lest we go off the rails.

large context limits is one of the things that make the open model competitors very attractive to me.

[MattRix]

If you want stuff to be referenced regularly, why are you doing it in the conversation instead of just putting the documents in the directory and referring to them in AGENTS.md or whatever?

[thih9]

What plan are you on?

With Anthropic I run out of quota very fast on a Pro plan with long contexts.

Is long context primarily for Max tier or for API usage? Or are there special ways of working with long context on a Pro plan?

[tekacs]

Yep, Max. I would definitely struggle to code with Anthropic's plans on Pro.

[thih9]

That helps - I'm struggling to code on Pro, it's useful to know that it's not just me. Thanks for answering!

[troupo]

> The lack of long context is the main reason that I still end up using Anthropic.

Anthropic's "long context" is mostly marketing bullshit. It semi-reliably holds about the same amount of info, and then starts suffering the same issues.

[nojito]

Why arent you using /goal?

[wwind123]

A typical suggestion is to make the agent create or update a .md file once in a while, so the agent would remember the new important things that have shown up since the last update. The problem is, how would the agent know which things are really important. If it knows that well, then /compact should work well too...