[ivan_gammel]

It might be not that bad. “Good enough” open-weight models are almost there, the focus may shift to agentic workflows and effective prompting. The lifecycle of a model chip will be comparable to smartphones, getting longer and longer, with orchestration software being responsible for faster innovation cycles.

[ACCount37]

"Good enough" open weights models were "almost there" since 2022.

I distrust the notion. The bar of "good enough" seems to be bolted to "like today's frontier models", and frontier model performance only ever goes up.

[ivan_gammel]

The generation of frontier models from H1 2025 is the good enough benchmark.

[ACCount37]

Flash forward one year and it'll be H1 2026.

[ivan_gammel]

I don’t see why. Today frontier models are already 2 generations ahead of good enough. For many users they did not offer substantial improvement, sometimes things got even worse. What is going to happen within 1 year that will make users desire something beyond already working solution? LLMs are reaching maturity faster than smartphones, which now are good enough to stay on the same model for at least 5-6 years.

[ACCount37]

Any considerable bump in model capability craters my willingness to tolerate the ineptitude of less capable models. And I'm far from being alone in this.

Ever wondered why those stupid "they secretly nerfed the model!" myths persist? Why users report that "model got dumber", even if benchmarks stay consistent, even if you're on the inference side yourself and know with certainty that they are actually being served the same inference over the same exact weights on the same hardware quantized the same way?

Because user demands rise over time, always.

Users get a new flashy model, and it impresses them. It can do things the old model couldn't. Then they push it, and learn its limitations and quirks as they use it. And then it feels like it "got dumber" - because they got more aggressive about using it, got better at spotting all the ways it was always dumb in.

It's a treadmill, and you pretty much have to keep improving the models just to stay ahead of user expectations.

[ivan_gammel]

> users report that "model got dumber"

I have seen this with ChatGPT progression from 4o to 5.2 applied to the newest model. Old prompts stop working reliably, different hallucination modes etc.

[m101]

If you’re running at 17k tokens / s what is the point of multiple agents?

[ivan_gammel]

Different skills and context. Llama 3.1 8B has just 128k context length, so packing everything in it may be not a great idea. You may want one agent analyzing the requirements and designing architecture, one writing tests, another one writing implementation and the third one doing code review. With LLMs it’s also matters not just what you have in context, but also what is absent, so that model will not overthink it.

EDIT: just in case, I define agent as inference unit with specific preloaded context, in this case, at this speed they don’t have to be async - they may run in sequence in multiple iterations.