[nojito]

>bring it in-house

People don't like to hear this but the open models just aren't good for end to end agentic workflows.

There are some very very good small open models that can excel in certain finite bounded tasks, but the foundational models are essential to building out agentic pipelines that actually work.

[lelandbatey]

Have you been using those models? I've been using a hand-rolled orchestrator with Mimo v2.5 (I seem to be paying $0.017 per million/tokens after their heavy caching) and it's been very impressive. I started with it in Opencode as a harness, then had it build its own micro-harness with stdlib-only Python, then used that to build a local stdlib-only Orchestrator with CLI and web harness, and now I'm using that for improving itself and now multi-project wider-ranging software. I talk to a steward who investigates and plans, then the plans are handed off to parallel worker agents who go through a work, test, interrogate, review, eval state machine for quality (all autonomously) with me at the end just reviewing the work or getting notified if the work items aren't progressing due to the workers getting stuck. So far the only "getting stuck" has been bugs/configs on my part, all at a pretty great quality bar, and at a price that makes me laugh at things like Opus.

I'm still using Claude at work (they're the only approved provider), but wow are the smaller models starting to SMOKE the big ones. At this point, all I'd consider paying out of my own pocket for is the lowest-limit Anthropic/GPT plan to get a big model as the Steward, but I wouldn't pay for ANY of the Anthropic models as the workers who do all the work. And as time passes, I don't know if I'd even do that; the open models are serving SO well.

[lifeisstillgood]

So you are using a “cloud” provider and at 1c per million tokens …

Love to hear more about how you structure the orchestrator etc

[lelandbatey]

Yes, it's a "cloud provider" but it's a cloud provider running an open model you can download (and that other cloud providers do host). I just happen to not have a computer big enough to host it.

As for the Orchestrator, it's pretty simple. In essence, it's like "Jira/Trello/Kanban on autopilot". Work items have states, a state machine defines how those work items transition between states, states are todo, in progress, retrying, reviewing, code reviewing, done. work items also have connections, allowing the LLMs to specify a dependency graph, and the dependency graph informs the dispatch order/parallelism, as well as when branches have to be merged. I talk to the steward, the steward has tool calls for interacting with all the data, and the orchestrator auto-dispatches all the work that comes in. I can generate work as fast as I can describe it to the steward, and that's usually the bottleneck.

So far I haven't had to deal with "how do you get the LLM to re-organize the work mid flight due to a worker finding something not accounted for by the planning", but I assume it'll come soon. The most complicated digraph I've tossed at it was 9 items and 4 layers deep. The kind of work I've given it hasn't been scoped large enough yet, so we'll see how it tackles that.

[lifeisstillgood]

Ok, so I have to try that.

How are you specifiying the graphs? Is this on github (I am still trying to move from concept to how to actually do it (plus Inhave only just woken up and need coffee :-)

[smartbit]

Leland is a CS student, that being said I agree the concepts and pricing are tempting. There are more of these Kanban for Agents eg Multica, PlateSpinner, KitKot, Kanbots.

[lelandbatey]

I was a student whenever I wrote my old bios but I've been a developer professionally for over 10 years at this point.

My inspiration was the mayor idea from Gastown, plus wanting to formalize the informal workflow I used with agents and Jira at $dayjob.

[TacticalCoder]

> People don't like to hear this but the open models just aren't good.

Stuff like the latest DeepSeek, Kimchi and GLM are used and loved by many people. It's not using an open model that is difficult: it's having the hardware allowing to do so. It's pricey and require technical skills.

That's why most people who are using (excellent btw) open-weight models are just renting compute online.

[nojito]

They just aren't good at agentic work.

Also risking it all for some distilled models is a recipe for disaster.

[HarHarVeryFunny]

All of the smaller models from anyone are distilled from larger ones. I assume you are just trying to disparage the Chinese models, but what you are actually saying is that people should only be using the largest non-distilled models, not smaller ones like Sonnet. I assume the upcoming Opus 5 will be distilled from Fable 5.

[himata4113]

GLM 5.2 is the first model that is competing with the frontier, everything before it existed I would totally agree with you.

[jst1fthsdys]

I disagree, they are that good at agentic work.

[citizenpaul]

>end to end agentic workflows.

There are probably 100 competing versions what this phrase might encapsulate. Could you elaborate more on which version you are using exactly?

My experience is that frontier models are only marginally better and not close to the cost/value of the open models which are anywhere from 10-100x cheaper. Perhaps I'm not doing "end to end agentic workflows?"

[lifeisstillgood]

>> just aren't good for end to end agentic workflows.

This is (apparently) the conceit of SteveYegge / GasTown - no model can cope unassisted so chunk it up, run it and if it falls over remember the exact place and restart, merging it all in

But that’s not my point.

I believe that software is a new form of literacy and just as all Companies and societies are literate now, in the future (tm) companies will run exclusively on software - AI developed software and those who go all out will have the sort of advantages the Catholic Church had over .. guilds?

Anyhow, that’s me being AI optimist. But writing the code is going to be a small part of that transition - almost everything to do with LLMs that is claimed amazing (Computer vision is something else) - almost everything people say we need an LLM is stuff you could have done three years ago but your internal politics just would not let you. Oh look we can see if our policies are being met (you could have written the policies in code and solved the whole problem)

Im struggling to get it out but - almost everything AI is proposed for is stuff a well run engineering firm coukd have taken on. A software literate firm could have done without AI is where firms are hoping AI will Get them

Imagine how far ahead real software literate firms will be - as long as they don’t burn their runway in tokens

Which is why, the right play imo is still buy in-house as much as possible, engineer around the problems and explore the phase space at marginal Cost.

Then and only then think frontier models.