Hacker News new | ask | show | jobs
by verdverm 147 days ago
As soon as you involve the agent and having it make tool calls, it is no longer deterministic

This is in fact the very reason I set out to build my own agent, because Copilot does this with their `.vscode/instruction/...` files and the globs for file matching therein. It was in fact, not deterministic like I wanted.

My approach is to look at the files the agent has read/written and if there is an AGENTS.md in that or parent dirs, I put it in the system prompt. The agent doesn't try to read them, which saves a ton on token usage. You can save 50% on tokens per message, yet my method will still use fewer over the course of a session because I don't have to make all those extra tool calls

1 comments

I think there’s a conflation here between artifact determinism and agent behavior.

LogicStamp’s determinism claim is about the generated context: same repo state + config ⇒ identical bundles. That property holds regardless of how or when an agent chooses to read them.

Tool calls don’t make the artifacts non-deterministic; they only affect when an agent consumes already-deterministic output.

LSP is equally deterministic in that regard and more token efficient

Can you address the token inefficiencies from having to make more tool calls with this method?

On tool-call overhead: in the MCP flow it’s typically 1–2 calls per task (watch_status once, then read_bundle for the relevant slice). In watch mode we also skip snapshot refresh entirely.

Token-wise, the intent isn’t “dump everything”. it’s selective reads of the smallest relevant bundles. If your workflow already achieves what you want with AGENTS.md + LSP querying, that may indeed be more token-efficient for many sessions.

The trade-off LogicStamp is aiming for is different: verifiable, diffable ground-truth artifacts (CI/drift detection/cross-run guarantees). Tokens aren’t the primary optimization axis.

I'm not sure the comparison I'm hoping to draw out is coming through

This seems more similar in spirit to AGENTS.md than LSP, so I'll make the comparison there. Today, I require zero tool calls to bring my AGENTS.md into context, so this would require me making more tool calls, each of is a round trip to the LLM with the current context. So if I have a 30k context right now, and you are saying 1-2 calls per task, that is 30-60k extra tokens I need to pay for, for every one of these AGENT.md files that needs to be read / checked to see if in sync.

I use git for the verifiable / diffable ground truth artifacts. I can have my LSP query at different commits, there is no rule it can only access the current state of the code

I think we’re optimizing for different constraints. If your goal is zero extra round trips and you’re happy with AGENTS.md auto-injection + LSP queries, then I agree LogicStamp won’t be a win on token cost for that setup.

The AGENTS.md comparison isn’t “same thing” - it’s a different layer. AGENTS.md encodes human intent/heuristics. LogicStamp generates semantic ground-truth contracts (exports, APIs, routes) from the AST so they can be diffed and validated mechanically (e.g. CI drift detection).

Git + LSP can diff/query source across commits, but that’s still a query workflow. LogicStamp’s goal is a persistent, versioned semantic artifact. If your workflow already covers that, then it may simply not add value - which is totally fine.

I think none of us know what we are doing with the new toy and are still trying to figure it out lol. So in some sense, there are a lot of ideas being offered, often in many shapes themselves. Just look to the agent sandboxing space

I still have a hard time seeing why I want something like this in my agentic or organic software development. I tried something nearly identical for Go, and having all that extra bookkeeping in context wrecked things, so on an experiential level, the custom DSL for giving the agent an index to the code base hurt overall coding agent performance and effectiveness.

What works far better is having very similar in content, but very curated "table of contents" for the agent. Yes, I also use the same methods to break it down by directory or other variables. But when it reads one of these, the majority is still noise, which is why overall performance degraded and why curation is such a difference maker.

Do you have evaluations for your project that it leads to better overall model capabilities, especially as they compare to a project that already uses AGENTS.md?

btw, I put this stuff in AGENTS.md now, you can put whatever you want in there, for example I auto generate some sections with Go's tooling to have a curated version of what your project does. I don't see it as a "different layer" because it is all context engineering in the end.