[sitkack]

I had to suffer with coworkers that would write doctests, bad docs and bad tests together at last!

The solution here is to write real tests and hyperlink them in a marked up form to the functions they actually test. Coding inside of a doc string is an epic troll.

This is a warning to anyone getting seduced by doctests, stay away! They invert the problem, when one should just write tests, it is a problem with the documentation and code discovery tools that makes this seem like a good idea.

[dbaupp]

I think there's another perspective on doctests, that seems more convincing to me: doctest functionality allows testing for the documentation, not a way of writing tests in documentation.

In particular, it's great if the documentation includes clear and simple examples for learning from, and it's even better if these are validated as working. This means that the focus of the code in documentation is typically different to a test (it doesn't need to include such precise validation or look at all the edge cases or regressions), but it's still really useful to have them automatically run.

[steveklabnik]

What is the distinction between a "real" test and a doc test that you're making? At least in Rust, they are the same thing.

[masklinn]

I'd guess their coworkers decided to write all tests as doctests, rather than use doctests to ensure examples run fine?

Python's doctests are also formatted and in some ways behaving like an interactive shell session, with

    >>> code here
    output here

rather than just "literate code" so the execution context is a bit strange. This makes complicated doctests hard to inspect and debug. Doubly so because there's almost no tooling which understands doctests.

And of course on the flip side the best tests make for absolutely terrible examples since they try to exercise weird corner cases.

[jedbrown]

I think it's between a test written inside a comment (where smart IDE features don't apply) versus `#[test]` in normal code, where you have a rich editing environment and instant feedback (including `cargo build` failing immediately on errors).

Perhaps it's just my perception, but doctests (in comments) seem slower to run.

[steveklabnik]

Ah; rust-analyzer syntax highlights doc tests (though not perfectly), but the higher order bit of "IDE features don't work as well" makes tons of sense, thanks!

[sitkack]

This is also part of my complaint. At least in Python, doctests limit the ability for the IDE to be effective.

They hinder the writer as well as the consumer, since they make it harder to write effective tests. What I am saying is, that all tests should be first class wrt the documentation.