[2001zhaozhao]

I had a bit of this impression when reading the post as well as the authors' product website. A lot of it does seem to be stuck in 2025.

For instance I think their post "long-context isn't the answer" on their website post straight-up isn't accurate, and gives me the impression they are just extrapolating previous performance to new models. In my experience, Opus 4.6 and newer have worked very reliably for long context to me (i don't perceive any intelligence drop at 700-900K tokens). Yeah it's extremely cost-inefficient, but it works.

[chmod775]

> i don't perceive any intelligence drop at 700-900K tokens

I have countless examples of this for Fable 5 and Opus 4.8 at just half that many tokens. Sometimes they can't even remember the correct workflow even if they're merely an orchestrator and have been going through the same 5 step loop multiple times for pretty much their entire context window.

Wall of shame with an Opus and a Fable oopsie from just yesterday: https://i.imgur.com/znSJ3OE.png and https://i.imgur.com/DLOHQ1B.png

One of them is particularly egregious because it uses a CLI tool to step through the stages that literally tells it what to do next. How do you fuck this up? Needless to say that tool was revised to force the agents to acknowledge everything happened, but I'm sure it will find a way to eventually mess up anyhow.

And orchestration should be the easy part. It's pretty much just writing and adjusting workflow files. Don't get me started on code quality - but I realize that's subjective and my bar is unusually high. There's a reason I have them run adversarial verifiers along multiple axis before their output is even worth my attention.

[hansvm]

FWIW, I notice the intelligence drop drastically with long contexts with Opus 4.6. It's barely usable for anything intricate and long. That long window is good for _something_, but it's not as good as a short window.

[Terretta]

for synthesis. read the entire whatever in one gulp and boil it down or plan what to do about it.

not for steps.

[hansvm]

Oh yeah, that's a use case I love long contexts for. "Here's a day of prod logs, what's broken?" It's an incredible utility, but the qualities much less than with shorter contexts, and you see it much more apparently when you do similar coding tasks with progressively larger windows.

[eru]

> Yeah it's extremely cost-inefficient, but it works.

And competition is fixing that as we speak.