[Tenoke]

Claude seems to forget what you tell it in very long work sessions (things that take weeks to develop), no matter how many times you tell it which part is extra important. I dont use goal (I guess I should), but presumably it makes it actually remember the most important instruction. I believe this here is about shorter sessions where the issue doesn't crop up as much.

[jrockway]

I put context size in my Claude Code status bar and ... it does get "tired" when it's at 50%-60% context. I figured this out when I'm like "now modify the testbed to run this and add a test to exercise blah blah blah" and it was just like "That would be a substantial infra lift." In my head I say "I am paying a substantial amount for you to do this!" but just type /compact and re-prompt and my "substantial infra lift" is done without complaint.

I guess everyone needs a nap after a long day of conversing and writing code. So like us!

/compact is prone to error and I wouldn't recommend it in the middle of work. But when you are switching to a related but not completely new task, it helps. ("Now write the integration tests." vs. "on foo.go line 476 that you just wrote, I think there is a deadlock with bar.go line 123 that you just added". It doesn't really need the context to write the tests, it can get that by reading the code. But for iterating on lines of code it just produced, /compact is going to throw away whatever "thought process" led to that code and it's usually not a great thing to do.)

[aytigra]

I've actually found that compacting as often as possible, after planning, then between each implementation step works the best, it both unloads unnecessary context of previous edits and test runs, makes thinking cheaper, and most importantly after each compaction it re-loads CLAUDE.md which make it much more enforcing (otherwise it just moves to the back of context and slips from model's attention).

[jrockway]

I am not sure whether I love the plan + code workflow, but when I do it, I do /clear and instruct "implement PLAN.md" or whatever. (Probably better to do /goal implement PLAN.md; I haven't tried it though.)

[aytigra]

Yeah, I should probably /clear more often. Compaction seem cheap by itself (all cached input and small output), but it often marks unnecessary files for pre-load.

[fendy3002]

Use untracked file as memory and instruct agent to put progress into that, it'll help managing context in current and new sessions

[Tenoke]

I do that but it ends up putting so much extra crap there that it has the same drawbacks.

[esperent]

This is one of the advantages of pi. I made a /protect command that protects the message from compaction. I also protect skills automatically.

So for long running tasks I'll do

    /protect your goal is...

[sdesol]

I am curious as to how this protect works. I am currently working on a pi brain extensions and one of the commands that I will support is:

/brains inspect

that will let you inspect a session and one of the features is the ability to review a compacted message like so:

https://gitsense.com/screenshots/inspect-pi-session-compacte...

https://gitsense.com/screenshots/inspect-pi-session-compacte...

https://gitsense.com/screenshots/inspect-pi-session-compacte...

With /compact in Pi, it creates a message that you can easily review and I am curious as to how '/protect' works.

One of the features that I am working on is to make it easy for agents to retrieve the exact message/event before compaction and I am curious if /protect is a deterministic process or if it is just instructions.

[kaelwd]

https://github.com/k0valik/pi-blackhole works pretty well for this, instead of a summary of a summary of a summary it gets a rolling log of important instructions and discoveries, and can /recall exact contents of a previous message or tool call if needed.

[esperent]

That looks interesting, have you found it useful in practice? I'm a bit burnt out on these wildly ambitious big plugins (headspace, RTK etc.) because in each case when I've done personal evals they come out worse than the default.

[kaelwd]

I haven't done a proper benchmark or anything but it isn't noticeably worse at least, and the instant compaction with background processing is nice for slow models.

[esperent]

> it isn't noticeably worse at least

That's what I thought about RTK and Headroom too but when I did some fairly basic evals across a small set of tasks (a matrix of RTK, Headroom, both, or off) it came back very clear that off was best. I didn't dig into why too deeply due to lack of time, but it looked like the token reduction per turn was being replaced by the model needing more turns.

That and building my own pi extensions made me realize there's a lot of subtly here. I can build a tool that on paper works perfectly but tuning the system prompt to get it used correctly takes time and evals.

The problem is, nobody is doing them, or not well, and not reproducibly.

This looks especially hard for something designed for very long running tasks. We'd need to set up one version using normal compaction, one using this, across several tasks, leave them running for hours (burning tons of tokens) and then repeat at least a couple of times. If we don't do that, we're just guessing.

[throw10920]

I'm also a fan of Pi - although, doesn't Claude code (and most other harnesses) already have "custom summarization instructions" for compaction/summarization that already allow you to do this? Or have you added on something extra?

[lionkor]

They way I read it, this is a workaround for pi, not for Claude.

Edit: removed off topic political spam

[throw10920]

Can you please keep the unsolicited ranting out of the replies? It breaks the guidelines and degrades the value of the site. Thanks!

https://news.ycombinator.com/newsguidelines.html

[lionkor]

It's part of the rationale for why one would choose Pi over Claude, even if Pi lacks the feature. I admit my tone is entirely inappropriate though.

[throw10920]

> It's part of the rationale for why one would choose Pi over Claude

It's unrelated to my direct comment and it's a generic tangent off the thread further up, which is about technical issues with Claude Code. As the guidelines say:

"On-Topic: Anything that good hackers would find interesting. That includes more than hacking and startups. If you had to reduce it to a sentence, the answer might be: anything that gratifies one's intellectual curiosity."

"Eschew flamebait. Avoid generic tangents."

"Please don't use Hacker News for political or ideological battle. It tramples curiosity."

Nobody wants to see you hijacking technical threads to turn them into political flamewars. It's against the guidelines, anti-intellectual, and profoundly boring.

[jswny]

Unfortunately I’ve used Claude and GPT models for a long time in a variety of harnesses and I agree with you and I think it’s the compaction.

For some reason, codex compaction is like black magic. I’ve never felt like I can just one one continuous thread with other models, Claude I carefully curate when I compact

[brookst]

Compacting at all is a mistake. With 1m context window there is no reason for a single task to require compaction.

Much better to spend tokens breaking the task into chunks, documenting and storing them durably, then executing each one in clean context and just /clear after.

It’s a similar concept to compaction, just planned in advance. Much much more effective, and doesn’t burn tokens and time (“wall-clock”, Claude) doing the compaction.

[wgjordan]

> With 1m context window there is no reason for a single task to require compaction

Only if money is no object. Cache reads are cheap (10% of uncached input costs) but definitely not free, and cached reads dominate session costs at long context lengths. A prompt at 20k context with $0.01 in cached reads would cost $0.40 in cached reads at 800k context, that quickly adds up for long sessions.

[brookst]

I’m not following the implication that these economics argue for jsing compaction instead of just clearing context?

[wgjordan]

Your 'with 1m context window' implied that some manually-curated task 'chunks' would overflow smaller context windows (eg 200-400k tokens). If you're instead curating chunks small enough to avoid getting burned on long-context cache-read costs, you're not using a 1m context window at all. At that point, compaction is a convenience over manually chunking and clearing context for fine-grained bits of work.

I think the stronger claim is: there is no reason for a single task to require a 1m context window.

[mrtesthah]

Most models’ reasoning abilities drops off significantly between the 256K-1M token ranges of the context window. There’s too much stuff to “pay attention to” at once.

[hedgehog]

I auto compact around 200k tokens both due to this and because the cached read cost really escalates when sessions have more tokens than that (too short and you pay a lot in per-compact re-reading of state)

[jeeeb]

Agree. Another consideration is that input token costs are effectively quadratic with the number of conversation turns. The minute you get into a long conversation you can see the costs shoot up.

You can get far more gas out of even a $20 plan of you’re careful to break things up into relatively small discrete steps, clear context regularly and give the model plenty of information to work with.

My workflow for bigger features is to write out a plan document and then proceed in smaller implementation steps, reviewing as I go. If I find something odd, I ask the agent why and often that leads to discovering a new dimension to the problem, which in turn is an opportunity to adjust the approach.

[kccqzy]

You just haven’t worked on tasks that are complicated enough. Occasionally it took more than 1M tokens just to come up with a plausible plan.

Personally I find using /rewind judiciously is better than using /compact. The latter essentially gives you no control of what details to discard, but the former at least has coarse-grained control.

[brookst]

Oh my goodness. I’ve used over 1B tokens on a single feature. I’m running at about 25B tokens/month right now.

My whole point was that by planning in advance you can shard the work into manageable sections with clear beginnings and outputs with acceptance croteria, and never compact, or even use more than a few hundfed thousand tokens in context.

It’s all hierarchical. Looking at an eval feature building right now, it’s 20ish build plans, each with zero to five or so /clear moments.

But maybe that’s the key thing… I don’t iteratively prompt ad hoc software writing. I do iterate on requirements, but if those are solid enough there is no “now write this function, now write that module”.

[irthomasthomas]

Have you ever looked at how much performance drops as context grows? The difference in intelligence between 100k and 1M is huge, like opus drops to haiku level performance, or worse. For that reason I try to keep under 200k. That feels about the upper bound for tasks requiring accuracy.

[tony_cannistra]

This is the way.

[weiliddat]

One interesting harness thing I saw Cursor do is to give the model access to the entire thread.

Even if it doesn't fit in the context window, the model can search through past turns and sanity check if something doesn't seem to be going right, or be prompted to follow an early message, "when starting on a new item, review the first message for how we should approach this"

[semiquaver]

Unlike other model/harness pairs, codex+gpt also passes an opaque encrypted artifact speculated to be an embedding representing the conversation back to the successor generation which is “denser” or at least higher fidelity than summarized text.

[perching_aix]

That's wild to read considering Codex's compaction to me feels like the model becoming demented. Super aggressive, throws away almost everything.

[jedisct1]

This is not an issue with Claude, but with Claude Code.

With swival.dev you can use long sessions without ever doing any manual compaction or reset, even with Claude models.

[what-the-grump]

Pretty much, I stopped using Claude for that reason, I shouldn't have to keep track of the context of the model all the time. And god forbid I don't, the amount of time I've lost to Claude just sending it and then lying is not worth it when 5.5 will just do stuff, compact 2-3 times in the task and still not go insane.

This whole 1 million context window is a lie after 300k Claude degrades to unusable, and compact doesnt help. I've had Codex tabs open for weeks, I have to regularly restart Claude.

[Androider]

Quality degrades long before you hit compaction, compaction is the "Next gas station, 100 miles" sign but you're already in the middle of nowhere.

You don't need to go down the rabbit hole of crazy workflows, but to avoid slop:

- Break down the work into tasks

- New context. Create a plan for one task.

- New context. Implement the plan.

- New context. /code-review the implementation

- New context. Fix the review findings.

- Repeat for next task.

I do this with Fable 5, and the quality is consistently quite good. If the context goes over 50%, the quality will become crap and you end up with 4 duplicates of the same thing across the codebase. Letting a current session review its own work is like asking a student to grade their own paper.

[beefsack]

For the vast majority of models I've worked with, 150k tokens of context seems like some magic spot. If you can stay under it then the session will stay on track and the agent won't start getting forgetful and weird.

Just like with real work, it's valuable to break down big tasks into small tasks that you can knock over in a single session. When a session does start getting too big, you just need to ask the agent to reply with a comprehensive handover report and paste it into a new session.

[robwwilliams]

For my quirky workflows I need 100K even to prep Opus 4.8 or higher to begin good work. Not programming, but a complex synthetic architecture for implementing more attentional autonomy for machines.

[IceDane]

Stop using compaction. Stop treating sessions as something you keep alive for weeks. Set up a proper workflow for handing off work and start handing off well before maximum context is hit. It's less efficient, slower and more costly to constantly be sitting at high context and compaction is just not good.

[robwwilliams]

There is no one-size-fits-all, even for programming.

[demosthanos]

There isn't, but everything that they said above is absolutely true and people who try to fight it will suffer. There is lots of flexibility within those constraints, but the constraints aren't imaginary.

[stevenhuang]

If you want to use LLMs effectively, everything OP said is true. It has been this way since the beginning. There will be a time when long context and compaction will be effective, but now isn't it.

[IceDane]

There's a good way and a bad way to use tools. My suggestions were based on my extensive usage of the tools in question.

[epolanski]

It doesn't forget, it doesn't care.

It's a great vibecoding model, but a crap assistant, and it forgets in few turns.

It's trained on benchmarks.

[michaellee8]

i found that at 700k-ish context even fable becomes an idiot, maybe openai's decision to cap codex context at 400k is correct, 400k is really a sweet spot where most part of the context is reliable.