Hacker News new | ask | show | jobs
by nsvd2 112 days ago
I think that recording dialog with the agent (prompt, the agent's plan, and agent's report after implementation) will become increasingly important in the future.
3 comments

I have this at the bottom of my AGENTS.md:

You will also add a markdown file to the changelog directory named with the current date and time `date -u +"%Y-%m-%dT%H-%M-%SZ"`, record the prompt, and a brief summary of what changes you made, this should be the same summary you gave the developer in the chat.

From that I get the prompt and the summary for each change. It's not perfect but it at least adds some context around the commit.

Isn’t the commit message a better place to add what and why? You might need to feed some info that the agent doesn’t have access to “we are developing feature X this change will such and such to blah blah”. The agent will write a pretty good commit message most of the times. Why do you need a markdown file? Are releasing new versions of the software for third parties?
Cheaper and faster retrieval to be added to the context and discoverable by the agent.

You need more git commands to find the right commit that contains the context you want (either you the human or the LLM burning too many token and time) than just include the right MD file or use grep with proper keywords.

Moreover you could need multiple commits to get the full context, while if you ask the LLM to keep the MD file up to date, you have everything together.

I doubt you can give more context to an LLM from a README file than 500 properly written commits. Or to a human for that matter.
The problem isn't giving MORE context to an agent, it's giving the right context

These things are built for pattern matching, and if you keep their context focused on one pattern, they'll perform much better

You want to avoid dumping in a bunch of data (like a year's worth of git logs) and telling it to sort out what's relevant itself

Better to have pre-processing steps, that find (and maybe summarize) what's relevant, then only bring that into context

You can do that by running your git history through a cheap model, and asking it to extract the relevant bits for the current change. But, that can be overkill and error prone, compared to just maintaining markdown files as you make changes

"You want to avoid dumping in a bunch of data (like a year's worth of git logs) and telling it to sort out what's relevant itself"

So instead you give it a years worth of changelog.md?

"Better to have pre-processing steps, that find (and maybe summarize) what's relevant, then only bring that into context"

So, not a list of commits that touched the relevant files or are associated with relevant issues? That kind of "preprocessing" doesn't count?

"You can do that by running your git history through a cheap model, and asking it to extract the relevant bits for the current change. But, that can be overkill and error prone, compared to just maintaining markdown files as you make changes"

And somehow extracting the same data out of a [relatively] unstructured and context-free (the changelog only has dates and description, that will need to be correlated to actual changes with git anyway...) markdown file is magically less error-prone?

How often, in your experience, do people read those auto-generated markdown files? Do you have any empirical data on how useful people find reading other people's agents' auto-generated files?
How often is it the same summary given to the developer in the chat?
Why doesn't this apply to human collaborators as well? If you need all this extra metadata to comprehend the changes, isn't that kind of going backwards? You spend time (setting up the agents, building extensive prompts that explain soooo much of how to do things, adding to whatever markdown file you think controls the parrot) and money (so many token$), to get code that you don't comprehend, and just decide to fill your repo with all of the above to... what exactly does all this accomplish? So you can later ask another parrot to "fix" something?
Agree, but current agents don't help with that. I use Copilot, and you can't even dump it preserving complete context, including images, tool call results and subagent outputs. And even if you could, you'd immediately blow up the context trying to ingest that. This needs some supporting tooling, like in today's submission where agent accesses terabytes of CI logs via ClickHouse.
I've had some luck creating tiny skills that produce summaries. E.g. a current TASK.md is generated from a milestone in PLAN.md, and when work is checked in STATUS.md and README.md are regenerated as needed. AGENTS.md is minimal and shrinking as I spread instructions out to the tools.

Part of my CI process when creating skills involves setting token caps and comparing usage rates with and without the skill.