[NitpickLawyer]

> another signal I've been thinking about and I'm seeing increasingly get brought up is the state space of a system

State space of a system AND the way to make it accessible / visible to a model. Many times a model can work magic if it can "see" the state of a system in a way that suits it. That's why sometimes having a cli added to the environment seems like such a big unlock. Because that cli usually takes a complex state and allows visibility into it, and possible manipulation in a structured way.

[nicoty]

I've been thinking a lot about this recently. In my case: how do I get an agent to see the important parts of the current plan and get it to stick to it without deviating, especially as loops get into longer and longer cycles and compactions erase prior context?

I think the problem with just encoding a whole plan in a single markdown file is that it gets polluted really quickly (agents can stop adhering to instructions to keep it clean and conform to a specific structure), which makes it harder for agents to see which parts of the plan they should focus on. As such, I've reached a similar conclusion about giving the agent access to CLI tools to help them deal with this. To try to mitigate this, I've been getting Claude to develop for me a CLI tool that:

1. Scaffolds reusable, structured plan templates and a reusable workflow that structures how to tackle the plan, step by step.

2. Validates that the plan files still conform to the correct, parseable structure.

3. Parses and evaluates the plan files, to determine what the current state is and what the next valid transitions and states are according to the workflow, like a state machine, and outputs instructions and reminders for agents as to what they should do at each step of the workflow.

So far, I've been dogfooding the tool and it seems promising: I can leave Claude running for longer and it doesn't drift as much. However I haven't ran any benchmarks yet and I'm still not entirely happy with the state of the codebase ( https://github.com/nothingnesses/agent-scaffold ), so take this with a grain of salt.

That said, I'm also bullish on using agents with formal methods and proofs. Type checkers and compile-time checking in general are great because they surface errors early and with great specificity. So if you can encode your specifications with, e.g. dependent types, you can use the type-checker as a way to steer the agent when it goes wrong and gets off-track.

[LiveTheDream]

You should take a look at OpenSpec[0]. It implements your points 1-3.

[0] https://openspec.dev/

[Vgoose]

Ive have a definition of state space that is calculated by just the types of the system. It’s a rough approximation of the true state space but it’s convenient (and more reflective of what we actually mean by system state IMO) and types are inherently accessible to the model. I recently put down some thoughts around working with types to improve communication w/ AI that kind of sets this framing up: https://www.alecvo.org/blog/types-with-ai/. The actual definition is still WIP.