[susam]

It is a bit weird to see LLMs suddenly being presented as the reason to follow what are basically long standing best practices.

'You must write docs. Docs must be in your repo. You must write tests. You must document your architecture. Etc. Etc.'

These were all best practices before LLMs existed and they remain so even now. I have been writing extensive documentation for all my software for something like twenty years now, whether it was for software I wrote for myself, for my tiny open source projects or for businesses. I will obviously continue to do so and it has nothing to do with:

> AI changes the game

The reason is simply that tests and documentation are useful to humans working on the codebase. They help people understand the system and maintain it over time. If these practices also benefit LLMs then that is certainly a bonus, but these practices were valuable long before LLMs existed and they remain valuable even now regardless of how AI may have changed the game.

It is also a bit funny that these considerations did not seem very common when the beneficiaries were fellow human collaborators, but are now being portrayed as very important once LLMs are involved. I'd argue that fellow humans and your future self deserved these considerations even more in the first place. Still, if LLMs are what finally motivate people to write good documentation and good tests, I suppose that is a good outcome since humans will end up benefiting from it too.

[seanwilson]

> It is a bit weird to see LLMs suddenly being presented as the reason to follow what are basically long standing best practices.

Maybe it's the speed of LLM iteration that makes the benefit more immediately obvious, vs seeing it unfold with a team of people over a longer time? It's almost like running a study?

I have a similar reaction to strong static types being advocated to help LLMs understanding/debugging code, catching bugs, refactoring... when it's obvious to me this helps humans as well.

Curious how "this practice helps LLMs be more productive" relates to studies that try to show this with human programmers, where running convincing human studies is really difficult. Besides problems with context sizes, are there best practices that help LLMs a lot but not humans?

[codethief]

Agreed. LLM usage just makes the impact more visible.

I bet one of the next "revelations" is going to be: Avoid sprinkling side effects throughout your code. Prefer functional code, push side effects to the boundary (functional core, imperative shell).

[forrestthewoods]

> It is a bit weird to see LLMs suddenly being presented as the reason to follow what are basically long standing best practices.

About 95% of the work needed to make LLMs happy is just general purpose better engineering. Units tests? Integration tests? CI? API documentation? Good example? All great for humans too!

I consider this largely a good thing. It would be much worse if the changes needed for Happy LLMs were completely different than what you want for Happy Humans! Even worse would be if they were mutually exclusive.

It's a win. I'll take it.

[hex-m]

Reminds me of the "semantic web". Making content machine-readable has positive side effects. (clear structure, reduced ambiguity, separation of data and presentation)

It's another example where the reason for better engineering is to make machines (search engines) happy.

[ronsor]

AI means that you cannot defer software design until you've written half code; you cannot defer documentation to random notes at the end.

It has the effect of finally forcing people to think about the software they're making, assuming they care about quality. If they didn't, then it's not practically different from an insecure low-code app or something copy-pasted from 15 year old StackOverflow answers.

[lelanthran]

> AI means that you cannot defer software design until you've written half code; you cannot defer documentation to random notes at the end.

> It has the effect of finally forcing people to think about the software they're making,

Ah, and all this time I was reliably assured that waterfall, design-upfront, was a broken process...

[kingstnap]

Single iteration waterfall is a broken process. You really need those late stage usage feedback signals unless your requirements were somehow captured by God.

[lelanthran]

> Single iteration waterfall is a broken process. You really need those late stage usage feedback signals unless your requirements were somehow captured by God.

Waterfall, as prescribed and as practiced, had feedback signals.

[palmotea]

> It is also a bit funny that these considerations did not seem very common when the beneficiaries were fellow human collaborators, but are now being portrayed as very important once LLMs are involved. I'd argue that fellow humans and your future self deserved these considerations even more in the first place.

The reason might have been a cultural regression. At least with documentation, it seems to have been much better and a bigger priority a couple decades ago.

At the start of my career, teams produced documents as a part of their work, and there were even technical writers on staff. Then agile hit, the writers were laid off, much of what little documentation that was created was kept in various work-tracking systems and wikis that were periodically replaced, often with little to no migration.

[basilikum]

> The reason is simply that tests and documentation are useful to other humans working on the codebase.

Including future you

[benatkin]

Well, it's timely because there's a docs platform that has surged in popularity, and it really is not a good idea for most of those who need technical docs to be using a SaaS that approximates Squarespace.

[starkparker]

As a tech writer, it's not surprising. All LLMs did was get PMs on the same page as TWs, devs, and support toward prioritizing it, because now it benefits feature development in the short term instead of the long term. They can put it on a quarterly review slide and in an OKR, and their bosses will jump up and down and squeal and give them a raise instead of shooting them in the face.

[jamie_ca]

I see this as not just internal API/architecture/code documentation, but product documentation too. We maintain internal docs about how our product is used for our support, implementation, and sales teams to reference.

Right now it's hosted externally (in our "blessed" knowledge base) but if it could be pulled into the repo, and we set an AI reviewer on every pull request to sanity check that if the changes we're doing have a material impact on the feature as described in those docs that it should be flagged (or changes proposed) that'd be a nice win for keeping them up-to-date, and it's easy enough to publish markdown as html or even script an update to the canonical site when we merge to main.

[sublinear]

I agree and would go one step further. The way people are now talking to LLMs to write code is the way we need them to plan and discuss in meetings with humans.

Everything regarding AI-assisted development is basically training wheels for the young people coming into the workplace.

[formerly_proven]

There's a pattern where people create AI-specific infrastructure for coding agents which is essentially instantly obsolete because it's pointless. Stuff like most MCPs (instead of just using a CLI), agent-specific files (CLAUDE.md, AGENTS.MD, github-instructions.NET etc.) etc.

> You should have a good, concise introduction to the codebase that allows anyone to write and test a simple patch in under 15 minutes.

Yeah, that's the CONTRIBUTING file.

[j45]

LLMs are making it more possible to maintain.

[ErroneousBosh]

> These were all best practices before LLMs existed and they remain so even now

Okay, so what, should I be moving my docs out of the repo or something?

How should I make it as hard as possible for LLMs to make any use of or suggestions about my documentation?

[pooploop64]

Lately I have seen a lot of things coming full circle like this in a way that always seems positive for humans as well.

Many doomers are running around saying the future is grim because everything will be made for AI agents to use rather than humans. But so far everything done to push that agenda has looked more like a big de-enshittification.

Another one is Model Context Protocol, which brings forth the cutting edge (for 1970) idea of using a standard text based interface so that separate programs can interoperate through it.

If the cost of having non-user-hostile software is to let AI bros run around thinking they invented things like stdin and documentation, I'm all for it at this point.

If any AI bros are reading this here's another idea. Web pages that use a mostly static layout and a simple structure would probably be a lot easier for AI to parse. And google, it would be really beneficial to AI agents if their web searches weren't being interfered with by clickjacking sites such as Pinterest.

[8note]

there's an implicit ownership change, from having technical writers own the documentation, to including it as part of the commit.

when things are tiny or resource constrained, the same people are doing each task regardless, but "technical writer" is a real job around documentation and manual writing, so there's at least sometimes some real decoupling between the code and outwards facing documents.

that also covers for cases where people can write code well, but who's english(or whatever the target documentation language is) is shaky at best.