[Klaster_1]

The article very much resonates with my experience past several months.

The project I work on has been steadily growing for years, but the amount of engineers taking care of it stayed same or even declined a bit. Most of features are isolated and left untouched for months unless something comes up.

So far, I managed growing scope by relying on tests more and more. Then I switched to exclusively developing against a simulator. Checking changes with real system become rare and more involved - when you have to check, it's usually the gnarliest parts.

Last year's, I noticed I can no longer answer questions about several features because despite working on those for a couple of months and reviewing PRs, I barely hold the details in my head soon afterwards. And this all even before coding agents penetrated deep into our process.

With agents, I noticed exactly what article talks about. Reviewing PR feels even more implicit, I have to exert deliberate effort because tacit knowledge of context didn't form yet and you have to review more than before - the stuff goes into one ear and out of another. My team mates report similar experience.

Currently, we are trying various approaches to deal with that, it it's still too early to tell. We now commit agent plans alongside code to maybe not lose insights gained during development. Tasks with vague requirements we'd implicitly understand most of previously are now a bottleneck because when you type requirements to an agent for planning immediately surface various issues you'd think of during backlog grooming. Skill MDs are often tacit knowledge dumps we previously kept distributed in less formal ways. Agents are forcing us to up our process game and discipline, real people benefit from that too. As article mentioned, I am looking forward to tools picking some of that slack.

One other thing that surprised me was that my eng manager was seemingly oblivious to my ongoing complains about growing cognitive load and confusion rate. It's as if the concept was alien to them or they could comprehend that other people handle that at different capacity than them.

[datsci_est_2015]

> One other thing that surprised me was that my eng manager was seemingly oblivious to my ongoing complains about growing cognitive load and confusion rate.

Engineering managers in my experience (even in ones with deep technical backgrounds) often miss the trees for the forest. The best ones go to bat for you, especially once verifying that they can do something to unblock or support you. But that’s still different than being in the terminal or IDE all day.

Offloading cognitive load is pretty much their entire role.

[bluegatty]

We don't have the right abstractions in place to support true AI driven work. We replaced ourselves but we don't have the tools to do '1 layer up'.

[enknee1]

Nailed it.

We desperately need a new set of abstractions for human- and AI-based knowledge.

I prefer humans-as-a-network of abstractions piloting an organic robot perspective. Sans mathematical framework, this is an unsatisfying claim, I know... But just hear me out.

This allows for extreme complexity between individuals and for language to act as a standard serial com channel with high dimensional abstractions embedded across words - a network of abstractions unto itself. Models of this network are embedded in books and 'live' in oral history.

LLMs, then, are just a much better model of the abstraction networks that span people through language (and often thought).

Notice that they're NOT people. And that we are actively developing network science to accommodate the complexities of inherent in examining both the real world and modeled versions of these networks.

As an example, the tools to layer up can be envisioned as more networks on top of these networks: reasoning and cognitive patterns are captured in recursive transformer-based LLMs. So a metacognative model might actively generate LoRA for each prompt.

Again, much math and research needed. But it's been a very useful set of abstractions this far.

[matsemann]

Learning has always been to write things down. Just reading it seldom sticks.

[RealityVoid]

Absolutely not. Learning has been to experiment with the things until you form a effective mental model of the thing. Writing things does ab-so-luetely nothing except make you feel good in the moment. Just like listening to a lecture without engaging with the subject matter deeper.

Writing things down is important for organisational persistence of information but that is something else.

[direwolf20]

Writing is better than reading, but doing is better than writing.

[shimman]

How does this apply to coding when the act of writing IS doing? Or do you mean like coding "on your own" versus following a tutorial for example?

[coldtea]

Means writing code (doing) vs writing documentation / plans / project architecture documents and so on.

[direwolf20]

Writing code is doing

[matsemann]

Choosing what to write down is making a mental model, extracting the core and thinking about the subject.

Seems to me you're just a bad note-taker that blindly writes things down, and for some reason decided to use that lack of knowledge in a tirade against me..

[0wis]

Not sure humanity learned nothing before the last 8000 years. It was just very slow. Maybe we will need new ways to learn

[nsvd2]

I think that recording dialog with the agent (prompt, the agent's plan, and agent's report after implementation) will become increasingly important in the future.

[slashdev]

I have this at the bottom of my AGENTS.md:

You will also add a markdown file to the changelog directory named with the current date and time `date -u +"%Y-%m-%dT%H-%M-%SZ"`, record the prompt, and a brief summary of what changes you made, this should be the same summary you gave the developer in the chat.

From that I get the prompt and the summary for each change. It's not perfect but it at least adds some context around the commit.

[atmosx]

Isn’t the commit message a better place to add what and why? You might need to feed some info that the agent doesn’t have access to “we are developing feature X this change will such and such to blah blah”. The agent will write a pretty good commit message most of the times. Why do you need a markdown file? Are releasing new versions of the software for third parties?

[Belphemur]

Cheaper and faster retrieval to be added to the context and discoverable by the agent.

You need more git commands to find the right commit that contains the context you want (either you the human or the LLM burning too many token and time) than just include the right MD file or use grep with proper keywords.

Moreover you could need multiple commits to get the full context, while if you ask the LLM to keep the MD file up to date, you have everything together.

[atmosx]

I doubt you can give more context to an LLM from a README file than 500 properly written commits. Or to a human for that matter.

[generallyjosh]

The problem isn't giving MORE context to an agent, it's giving the right context

These things are built for pattern matching, and if you keep their context focused on one pattern, they'll perform much better

You want to avoid dumping in a bunch of data (like a year's worth of git logs) and telling it to sort out what's relevant itself

Better to have pre-processing steps, that find (and maybe summarize) what's relevant, then only bring that into context

You can do that by running your git history through a cheap model, and asking it to extract the relevant bits for the current change. But, that can be overkill and error prone, compared to just maintaining markdown files as you make changes

[mort96]

How often, in your experience, do people read those auto-generated markdown files? Do you have any empirical data on how useful people find reading other people's agents' auto-generated files?

[addaon]

How often is it the same summary given to the developer in the chat?

[just6979]

Why doesn't this apply to human collaborators as well? If you need all this extra metadata to comprehend the changes, isn't that kind of going backwards? You spend time (setting up the agents, building extensive prompts that explain soooo much of how to do things, adding to whatever markdown file you think controls the parrot) and money (so many token$), to get code that you don't comprehend, and just decide to fill your repo with all of the above to... what exactly does all this accomplish? So you can later ask another parrot to "fix" something?

[Klaster_1]

Agree, but current agents don't help with that. I use Copilot, and you can't even dump it preserving complete context, including images, tool call results and subagent outputs. And even if you could, you'd immediately blow up the context trying to ingest that. This needs some supporting tooling, like in today's submission where agent accesses terabytes of CI logs via ClickHouse.

[gopher_space]

I've had some luck creating tiny skills that produce summaries. E.g. a current TASK.md is generated from a milestone in PLAN.md, and when work is checked in STATUS.md and README.md are regenerated as needed. AGENTS.md is minimal and shrinking as I spread instructions out to the tools.

Part of my CI process when creating skills involves setting token caps and comparing usage rates with and without the skill.