Hacker News new | ask | show | jobs
by nsajko 1587 days ago
> rather than hard-coding inputs you simply always generate them

Why not both?

In any case, a rudimentary implementation is actually in Go's standard library already for a long time: https://pkg.go.dev/testing/quick@master

Though, note:

> The testing/quick package is frozen and is not accepting new features.

3 comments

> Why not both?

Absolutely fine and often good enough, these tests are typically very simple and easier to reason about, and make more sense in a whole range of situations (eg regression tests).

However, I would take a single input fuzzing test over one that uses hard coded inputs.

> Why not both?

Go's fuzz tester takes this approach. When a failing input is found, it's added to the source code directory with the intent of you checking it in.

testing/quick is not coverage-driven and so not good at truly "interesting" input cases. It's OK for simple invariants (e.g. anything arithmetic with an inverse) but I would not trust it to tell me anything interesting about a parser. At this stage I might even rely on the fuzzer to test simple invariants because the tooling is nicer.