[mrothroc]

I've been running a multi-agent software development pipeline for a while now and I've reached the same conclusion: it's a distributed systems problem.

My approach has been more pragmatic than theoretical: I break work into sequential stages (plan, design, code) with verification gates. Each gate has deterministic checks (compile, lint, etc) and an agentic reviewer for qualitative assessment.

Collectively, this looks like a distributed system. The artifacts reflect the shared state.

The author's point about external validation converting misinterpretations into detectable failures is exactly what I've found empirically. You can't make the agent reliable on its own, but you can make the protocol reliable by checking at every boundary.

The deterministic gates provide a hard floor of guarantees. The agentic gates provide soft probabilistic assertions.

I wrote up the data and the framework I use: https://michael.roth.rocks/research/trust-topology/

[peterbell_nyc]

Exactly this. I'm writing my own little orchestrator and memory system and because I have a modest number of workflows, I'm taking the time to specify them deterministically, describe them as a DAG (with goto's for the inevitable loops) and generate deternministic orchestration code. I'm trying to make most of the tool calls as clear and comprehensive as possible (don't make Opus convert a PDF, have a script do that and give it the text instead) and I'm putting all the things you'd expect to track state and assume ~20% task failure rate so I can simply wipe and repeat failed tasks.

Small model and (where still required) human in the loop steps for deterministic workflows can solve a surprisingly large number of problems and don't depend on the models to be consistent or not to fail.

Just invest heavily in adversarial agents and quality gates and apply transforms on intermediate artifacts that can be validated for some dimensions of quality to minimize drift.

[mrothroc]

I did the same with my own orchestrator. That's where I get my data.

It's amazing the power a simple workflow with automatic gate enforcement brings to agenting coding.

[binyu]

Interesting perspective. While the analogy may be somewhat intuitive, distributed computing exhibit a wider and more diverse set of challenges imo.

Example: Synchronization in naturally async environments, consensus, failure-safe system, etc.

[mrothroc]

Agreed that full consensus is overkill.

But I think the coordination problem is subtler than version control implies. In the (plan, design, code) pipeline they aren't collaborating on the same artifact. They're producing different artifacts that are all expressions of the same intent in different spaces: a plan in natural language, a design in a structured spec, code in a formal language.

Different artifacts which are different projections in different Chomsky levels but all from the same thing: user intent.

The coordination challenge is keeping these consistent with each other as each stage transforms the prior projection into the new one. That's where the gates earn their place: they verify that each transformation preserves the intent from the previous stage.

[bmitc]

How did you launch, orchestrate, and run the agents? Did you build your own framework, or are you just using the various CLI tooling?

[mrothroc]

I created my own framework. Long ago it started as shell scripts that I used in conjunction with aider. It was a very manual process.

It's grown over time to be a full MCP and CLI with stages and gates defined in YAML. I was thinking about open sourcing it but since the code grew organically I would need to do extensive cleanup to make it presentable.

But I do walk through the process on page 9: https://michael.roth.rocks/research/trust-topology/#9