[RandallBrown]

One of the main bullet points on the page is automated tests.

How do you write automated tests for documentation? Somehow require that blocks of code have documentation linked to them?

[thesuperbigfrog]

>> How do you write automated tests for documentation? Somehow require that blocks of code have documentation linked to them?

It could be tests to ensure documentation "builds" into all of the desired formats (e.g. web, pdf, ebooks, etc.) correctly.

Some programming languages have the idea of "documentation tests". In Rust, tests that are part of the documentation will run as part of the documentation build:

https://doc.rust-lang.org/rustdoc/write-documentation/docume...

[Nzen]

If we treat specifications written in gherkin syntax [0] as documentation, then the cucumber framework can match a line or stanza of gherkin to a test function [1].

I admit that, while I write instructions for how to test specific functionality in gherkin, our company would not countenance publishing a non-narrative description of the system's behavior to our client's employees.

[0] https://www.manning.com/books/writing-great-specifications

  Given a work order xx
  and xx isExpedite
  When an operator prints the jobcard
  Then expect a label in the footer that says Expedite

[1] https://cucumber.io/docs/cucumber/step-definitions/?lang=jav...

[darknavi]

Loads of things!

- Making sure example snippets still compile

- Checking if links are dead

- Check for standardized/proper formatting

Basically anything you'd want to enforce manually, try to enforce with CI.

[0cf8612b2e1e]

I believe Rust and Python have the ability to run tests defined in docstrings.

[Spasnof]

Yes there are certain libraries that can handle this. Essentially asserting that functions documented are valid / return the proper results.

See https://docs.python.org/3/library/doctest.html#module-doctes... as an example.

[simonw]

I wrote about my way of doing that here: https://simonwillison.net/2018/Jul/28/documentation-unit-tes...

Short version: have tests that use introspection (listing functions and classes in a module, iterating over JSON API endpoints in the codebase etc) and then run regular expressions against your documentation searching for relevant headings or other pre-determined structures.

[tpoacher]

In matlab, where functions are "always" defined in their own file, there is a tool that checks if function documentations have all the right headers expected conventionally by matlab's documentation system (e.g. header, usage, examples, see-also links, etc). So this would be one example, I guess.

But I too would be interested to hear other people's insights who subscribe to this Docs as Code model.

[spondylosaurus]

Linters like Vale are pretty common for docs(-as-code).

[blowski]

> Somehow require that blocks of code have documentation linked to them

The Symfony (PHP) framework now does this. Code and config examples in the docs have automated regression tests.

[brobdingnag_pp]

Or just require file/function level comments. Requiring them to be helpful can be managed interpersonally, like someone was slacking off (they are)

[fmbb]

Yeah that’s one way. And you can test that docs don’t link to code that does not exist.

Here are some other Good Ideas in a blog post I stumbled upon the other week: https://azdavis.net/posts/test-repo/