Hacker News new | ask | show | jobs
by Yokolos 23 days ago
This is the most frustrating part. You do everything you can to ensure there are clear instructions, you can keep the agent MD as concise and clear and short as possible. It still feels like it's all just a suggestion, and of course it is, because it's all just another part of the prompt.
1 comments
maccard 23 days ago
I’m in the “AI can be great, but it’s not right now” camp. I think that pulling the verification into the harness and having the harness execute it rather than the agent would genuinely make AI go to usable for me. But even prototyping a custom harness requires API billing which is just so expensive…
link
DCKing 23 days ago
You can put many agent constraints in precommit hooks if they're static checks. I ask agents to make commits, and e.g. in a Python project have the precommit hook fire off type checks, linting and even architectural things like import boundaries (using `tach`). When an agent is prepped to make commits themselves, it will catch pre-commit failing and correct itself. The existence of static checks themselves might also help agents gain awareness of the overall verification flow including larger things like tests, but that's hard to say for certain.

Putting structural code checks in a precommit hook is arguably better than pulling it into the harness, as it will enforce those constraints no matter whether an agent or human is making the commit.

link
maccard 23 days ago
> , as it will enforce those constraints no matter whether an agent or human is making the commit.

Unless the agent, or the human don't enable the precommit hooks in the first place.

link
Muromec 23 days ago
You don't need the pre commit hooks if you can make the harness hooks.
link
QuercusMax 22 days ago
Belt and suspenders. What if you make a manual code edit? What if your harness hook malfunctions?
link
maccard 22 days ago
What if you don’t enable pre commit hooks?

> what if your harness hook malfunctions?

That’s a bug in the harness and should be enforced

link
Muromec 23 days ago
Writing a custom harness was like 10 bucks of the communist model tokens last month and I ended up with an a actor library, a thing that has a stable identity, writes sutras and knows it wrote the harness itself. Weird.

Adding an auto runner of the unit tests is just... Boring?

link