[pmmucsd]

Related (maybe the same thing): Whenever an agent is planning there are often architecture and product choices that it asks humans to make. None of this intent is captured in the code or comments. We started a decisions.md file and updated CLAUDE.md and AGENTS.md to create an entry in the decisions.md file everytime it has to ask a human about what to do. It captures the intent so at least we have doc that describes why certain choices were made.

[nostrademons]

Human coders have the same problem too - oftentimes the most important question that future maintainers have of the code is "Why was this decision made?", but that's not captured anywhere in the code itself.

The right place for this is usually in the design doc or commit message, and robust engineering organizations will ensure that commits are cross-referenced back to design and requirements docs so you can trace decisions from git blame back to the actual rationale.

The same process also works pretty well with LLMs. Google, for example, is internally championing a process where the engineer has a dialog with the LLM to generate a design doc, oftentimes with an adversarial LLM to poke holes in the design. Once the design is fully specified, the last step is to ask the LLM to turn the design doc into code. This creates a human-readable artifact that traces the decisions that the human and AI collaboratively made, which then can be traced back from the code.

[irq-1]

> Google, for example, is internally championing a process where the engineer has a dialog with the LLM to generate a design doc ...

Smart. They'll be able to regenerate code in the future, with better LLMs. It also lets them redo the architecture combined with other parts of the system as context grows.

[CharlesW]

> Whenever an agent is planning there are often architecture and product choices that it asks humans to make. None of this intent is captured in the code or comments.

It must be in order to have an effective AI-assisted software engineering workflow, but it's not something that's magically done for you (usually). Knowledge management of project context is everything. Even the little music app I'm building has >100 ADRs (https://en.wikipedia.org/wiki/Architectural_decision), and that's just one form of project memory.

[sanderjd]

In my experience, those decisions generally (without additional prompting) end up in the PR descriptions, and then in commit history. Which is how it was with human written code as well...

Or if it's more product level decisions than technical, those conversations tend to be documented in tickets. Which, again, that's how this already worked.

[alexhans]

To me, design files living in the repo are what captures these high and low level decisions. They should express intent which is paramount for automation.

Where are you keeping your design? If it's a WYSIWYG doc like word, what would it take for you to switch to a more text driven world?

[crooked-v]

The structure https://acai.sh/ uses feels to me like a decent starting point for this kind of thing that avoids the "single giant file" issue.

[rhgraysonii]

I have a tool for this https://deciduous.dev

[hombre_fatal]

(The comment made me start hashing out some early stage thoughts here)

Intent is something that is already sorely missing in most software projects.

At best you might get good commit messages, or messages that link back to good issue/PR discussion.

But something I realize is that /implementation/ intent is also important, not just higher level "why are we here in the first place" intent.

We tend to cargo cult the idea that you shouldn't have to explain what you're changing because it can be derived from the change. e.g. Commit messages should only explain the why since the what is redundant.

But the what is not redundant. Most bugs are a desync from the what, since you obviously didn't mean to implement the bug. And you didn't mean to use UUID v1 instead of UUID v4 for that session token.

Now that my workflow is centered around generating plan files instead of doing the implementation myself, the goal is specifying the plan with enough granularity that anyone could implement the intended change when given the plan.

It's data I wish we always had in a software project, but it would involve way too much work and energy to do this level of planning before every change until the LLM age.

Now that implementation is automated, we live at this specification step. And it's really the specification/plan that I want to see first rather than the code.

I think a great way to approach this to change the PR / code review step to center around revising a markdown plan file until you can get the intent and specification right. That makes a lot more sense than only reviewing the code and the deriving the why and the intended what.

e.g. For my own projects, I'd rather someone submit a plan file as a PR that we can revise together, not code. The code is a derivation of the plan. And when code is derived from a plan, then the plan must contain all of the knowledge that is applied to (or missing from) the code since it's the source of truth.

Compare that to the status quo where you start with a PR code change and then kinda work backwards to synthesize the why and intended what, then see if the real what synchronizes with intention.