[Mitchem]

While I am skeptical of the results here, I am very excited for this new trend of making models faster. Running capable models at 1k TPS is more valuable for me than running better models at 30 TPS. I can only imagine the trend continues to move from "let's only make models smarter" to just incremental intelligence gains but with step improvements in speed.

[holoduke]

Indeed. For me opus 4.8 is good enough. If only it would be 100 times faster. You could run it in self verification loops much much faster. It sometimes takes 15 minutes for me to complete a simple task. For example configuring AWS agentcore and deploying an agent on it. Takes forever with Claude with constant issues it tries to solve.

[lnenad]

Why? I'm personally on the opposite end. Less babysitting/higher quality means more time goes back to me/the user. 1000tps of bad code means you have to keep validating the output and circling back.

[anthonypasq]

id rather iterate multiple times than wait 15 minutes to notice it made a mistake.

[lnenad]

Again, my point is exactly the opposite. Higher quality implies a mistake isn't made in a significant % of cases.

[unshavedyak]

It's a lossy conversion though. "Mistake" is relative to the stated goals and specifications which are often heavily lacking. So unless you write with a high degree of architectural and implementation specificity then it might make very high quality code that is still not what you wanted.

[lnenad]

You can ask for a complete feature/app/business. Or you can split up the work into verifiable/testable pieces and rely on a high quality AI to deliver. As time goes by the pieces will get larger as capability grows. I still trust myself and my experience when arch is involved, but AI has been great at tackling lower level stuff. And with Fable I don't really care it takes a while for it to complete, as I know I can trust it a lot more (which is what I personally prefer). Yes, with a 10k tps model you can iterate quickly. But that's not me personally.

[RussianCow]

But some requirements you don't realize you have until you start building. With a fast model, you can surface those really quickly and have more time to iterate and explore different solutions. With a slower but smarter model, you just hope that what it produces after an hour is what you were imagining.

And yes, with Fable, the chance of that is higher than with SWE/Composer, but in my experience it's not so much higher that the extra time and cost is worth it. But it certainly depends on your goals and what you're building.

[aunty_helen]

High tps is good for deeper agent thinking loops and openclaw etc. I was running cerebus recently doing some data heavy tasks, it managed to crash the server I was submitting posts to. 6 hour task down to ~1hr

[unshavedyak]

So i agree with you, but there's no SOTA model that i don't have to babysit. I'm not going to just throw a large pile of code in there unreviewed, and so what i want is faster iteration on code in logical, reviewable chunks. Ie just like i'd normally write myself; small, logical commits.

Faster iteration means i mentally checkout less and am more involved with the code being created.

My hope is that in the far far future, we can get LLMs so fast that i can work in my IDE like normal and the LLM will just be an extension of autocomplete. I can state a goal, rough out functions, code, etc, and it'll just work around me like a very fast pair programmer / autocomplete.

The chat interface is an intermediate step that frankly i hate. The faster it is the less i wait.

Now for vibe-slop i'm making on the side, yea i don't care about speed. But that's not something i'm employed to do or anything i truly care about. It's a different workflow entirely.

[lnenad]

I get it, you just prefer to do things differently

> Faster iteration means i mentally checkout less and am more involved with the code being created.

This is a good point I didn't consider and you're right. More interaction brings you closer to the code.

I still think that this is the opposite of what I personally want. Either I write the code (or a large majority of it), and be fully involved; or be more disconnected but more free to focus on other things. The middle ground removes me from the equation, but also requires me to babysit.