[xorcist]

Love your blog, but in this case I want to take a more nuanced, if not opposite, stance:

There are many things closely related to code, that shouldn't necessarily live in the same repository. First, we need a common understanding of what should live together in a repository. This is much like the discussion about mono vs. multi-repo. A good rule of thumb is that if it is branched together, it lives together.

Effective documentation is not only a strict API reference, and not something that can be generated from docstrings alone. It offers a high level overview to understand the problem being solved, the architecture of the software, and a general roadmap of how it is developed. Effective documentation should cover both backwards and forwards revisions and how those migrations should be handled.

But this is also true on a reference level. Reading the documentation of a specific function I want to know if something relevant happens to this function in the next revision. There is nothing worse than checking out documentation for current production revision 34.5 and follow best practice there only to discover I should have checked out revision 34.6 instead because best practice changes there. Specific revisions should be documented, but documentation should not be limited to a specific revision.

There is a scale of how closely other artifacts follow code revisions: Tests is mostly branched with code, and should probably live together. Documentation can sometimes be branched with code, some should and some shouldn't live together with code. Deployment code and configuration management must be able to deploy old and new code from the same code base, and is even less likely to benefit from living with it. Then there's application state and test data which is something else entirely.

[simonw]

If the deployment code needs to be able to ship different versions, I would keep that deployment code in a separate repository - with its documentation bundled there.

The other form of documentation that I am passionate about is documentation that lives in issues, and then linked to from commit messages.

The great thing about issues and issue comments is that they have a clear timestamp attached to them, and there is no expectation that they will be kept up-to-date in the future.

This makes them the ideal place to keep documentation about how the code evolved overtime, and the design decisions that were made along the way.

[xorcist]

That is also true. But I realize the above comment could be more clear, perhaps with an example.

A well working project such as git has a Documentation directory in the same repository. That's good, but that documentation is far from enough. The most canonical documentation is the "Pro Git" book. That documentation describes not only how to use the software, how versions differ and how functionality has evolved, and the what the internal data structures look like.

That documentation does not live in the git repository, and that's a good thing, as it is not versioned in the same way. That probably goes for a lot, if not most, of good documentation out there. Insisting on keeping documentation in the main code repository would go against that.

[simonw]

Sure, there's a whole world of documentation that can live outside of the repository - anything written by people outside of the core development team such as tutorials, books etc.

Of course, the problem with documentation like that is that it goes out of date almost by its very nature. The great thing about documentation in the official repo is that it can come with a guarantee to be maintained in the future - if that documentation gets out-of-date it's a bug, and should be fixed.

External tutorials and books carry no such expectation.

[xorcist]

Yes, but that Pro Git is developed by people outside the core development team (whatever that might mean) is beside the point. The point is that it is documentation that does not move in lockstep with the software. And most good documentation doesn't!

Had Hamano or Torvalds written Pro Git, it would still have been worse off had it been forced into the release schedule of git itself. The most useful documentation describes all versions of the software, and should be only loosely coupled with it. The same can be said for web sites for software which is also a type of documentation.

(This is, incidentally, also why over-reliance on docstrings and documentation testing makes good documentation hard. Certain examples need to be produced by older revisions of the software, especially when incompatibilities are what needs to be documented.)

Not all documentation is like that, of course, but when someone successfully insists on hard coupling documentation to code, that puts a hard limit on the type of documentation that will be written.

Despite how much having common release process for code, documentation, and deployment code tickles our nerd fancy, we should consider the opposite, as there can be benefits from a looser coupling. Never let smart stand in the way of good.

As perhaps is obvious, I too have fought the same hill many times, but from another perspective. Docstrings are good. Documentation in the code repository is good. But that is only a small subset of all documentation. Blessing that subset as canonical, or insisting that should be all there is, is a much too common mistake.

[simonw]

"Not all documentation is like that, of course, but when someone successfully insists on hard coupling documentation to code, that puts a hard limit on the type of documentation that will be written."

I don't think I've ever seen a project argue so passionately for "all documentation lives in the same repo as the code" that people were put off writing books or tutorials that didn't go in that repo.

I'm pretty sure we aren't actually disagreeing here. I'm fine with "unofficial" documentation - books, tutorials etc - that lives outside the repo. The official reference documentation that's updated to reflect changes made to the project should live alongside the project itself.

[NomDePlum]

"Specific revisions should be documented, but documentation should not be limited to a specific revision."

It's unclear to me what this is trying to argue. So apologies if the below entirely misses your point.

Technical documentation that refers to a codebase should live and be maintained with it. Otherwise there will certainly be drift. It can obviously still happen but at least it is provable it shouldn't have.

Not maintaining accurate documents is like disabling tests because they don't pass. It's easy to do but not right.

A checked in codebase to me should be as current and correct as possible. That includes accurate documentation.

I've rarely seen documentation that isn't tied to the codebase being maintained/valued.

[gcarvalho]

> "Specific revisions should be documented, but documentation should not be limited to a specific revision."

I think this boils down to “what do you do if you realize the documentation for v1.1 says that some feature does X when it actually does Y, but you’re already on version 2.2?”

If v1.1 docs are tied to the version tag in VCS, that incorrect statement cannot be fixed.

And it seems that fixing that forces you into backporting documentation even if you don’t release and maintain parallel versions of your software, which… kinda sucks.

To be honest, I much prefer docs in the repo, because it facilitates code review — a good patch touches some implementation, some tests, and some docs.

The downside when only the latest few versions of the software are supported and only the very latest docs are maintained is that historical docs will probably not be fixed.

[bluGill]

> If v1.1 docs are tied to the version tag in VCS, that incorrect statement cannot be fixed.

It can be fixed, just checkout the branch, git cherry-pick the updated change set, or even write it by hand, then do a re-release. You should have a process for this anyway as there might be a critical bug in that code that needs to be fixed and a re-release must happen.

Of course git's handling of branches leave much to be desired (I want mercurial to come back just for it's branch handling) and so developers often forget they can do this and it isn't really that hard. It is tedious though, and you will eventually have dual maintenance where you have to write the same code twice just because the two branches have diverged - this shouldn't be an excuse not to do it though.