[maccard]

I’m in the “AI can be great, but it’s not right now” camp. I think that pulling the verification into the harness and having the harness execute it rather than the agent would genuinely make AI go to usable for me. But even prototyping a custom harness requires API billing which is just so expensive…

[DCKing]

You can put many agent constraints in precommit hooks if they're static checks. I ask agents to make commits, and e.g. in a Python project have the precommit hook fire off type checks, linting and even architectural things like import boundaries (using `tach`). When an agent is prepped to make commits themselves, it will catch pre-commit failing and correct itself. The existence of static checks themselves might also help agents gain awareness of the overall verification flow including larger things like tests, but that's hard to say for certain.

Putting structural code checks in a precommit hook is arguably better than pulling it into the harness, as it will enforce those constraints no matter whether an agent or human is making the commit.

[maccard]

> , as it will enforce those constraints no matter whether an agent or human is making the commit.

Unless the agent, or the human don't enable the precommit hooks in the first place.

[Muromec]

You don't need the pre commit hooks if you can make the harness hooks.

[QuercusMax]

Belt and suspenders. What if you make a manual code edit? What if your harness hook malfunctions?

[maccard]

What if you don’t enable pre commit hooks?

> what if your harness hook malfunctions?

That’s a bug in the harness and should be enforced

[Muromec]

Writing a custom harness was like 10 bucks of the communist model tokens last month and I ended up with an a actor library, a thing that has a stable identity, writes sutras and knows it wrote the harness itself. Weird.

Adding an auto runner of the unit tests is just... Boring?