[ianbutler]

It's nice to see a strong long context open weights model that is multi-modal.

There are many applications that will benefit from the strength in audio here and until z.ai and co work in visual this could be very strong for general agentic applications, though I see there's a bit of weakness in the benches for areas that might make that less true.

Like all models need to slap it in your harness and do proper evals on the tasks you care about.

[0xbadcafebee]

MiniMax M3 and DeepSeek v4-Pro are highly capable long context open weight multi-modal models. But long-context is a trap, because performance still falls dramatically after 150k-200k context.

[ianbutler]

> But long-context is a trap, because performance still falls dramatically after 150k-200k context.

I often see this repeated, and it is not true task to task. I work on this daily and we have several tasks where long context is advantageous and our evals against a whole battery of models with different windows show it as being so.

This is why having good evals for the tasks you're working on is so important.

I do grant it's a good rule of thumb.

[subscribed]

That might be me getting paranoid but I actually see it above 200k (with Opus 4.8), but more often and MUCH more pronounced in the afternoon UK time.

[InsideOutSanta]

> But long-context is a trap, because performance still falls dramatically after 150k-200k context.

I'm not sure exactly what causes the difference, but this heavily depends on the model. In my experience with Opus 4.8, I can go well over 500k and still get extremely good results. A drastically different example was GLM-5.1, which worked great until about 100k and then turned insane almost immediately. They did fix that with 5.2, though.

[subscribed]

I'm an amateur so it influences my setup a lot, but Opus 4.8 above 250k context in my experience with planning and implementing its own plans gets much dumber than fresh Sonnet 5, to the point of forgetting / ignoring things in the last prompt, forgetting half of the convention for the (very small and simple) codebase, etc.

[gunalx]

5.1 going insane was probably also a inference quirk. Because it sometimes remained coherent the entire 200k context length.