| >Sure, but one thing I don’t understand is why fuzzing is not used more often for testing basically any pure function [...] Agreed 100%, and is actually what we encourage people to do. Since this is an intro article though, we wanted to keep things simple, and everyone understands the danger in accepting inputs over a trust boundary.
Your suggested method is what the fuzzing community calls differential fuzzing [0]. It’s been incredibly effective at finding bugs in crypto libraries [1], and is currently being used to fuzz different Ethereum node implementations [2]. There are other ways you can fuzz functions, and we sort of hint at this in the post when we say: “If you can define a property that must hold true for any given input (also called an invariant), then the fuzzer will look for inputs that break your invariant”. Usually this translates into writing assertions the same way you might when you’re writing property-based tests [3]. In fact, I think the fuzzing community has a lot to learn from property based testing. These are more advanced topics though, that we hope to cover in a later post, and why we omitted these details from this one. >This list of issues seems sort of manufactured [...] Developer friendliness means different things to different people depending on their area of expertise, years of experience, or interest levels. While the list may seem manufactured, we’ve found that unfriendly tooling and uncertainty about what to tackle first can turn developers off even trying to write a fuzz test at all. Understanding what makes a good fuzz test, instrumenting your code properly, running many fuzz tests at scale, and triaging and interpreting the results of a fuzzing run can make fuzzing prohibitively difficult for a new engineer to set up. This is what we’re focused on solving. >Advertising a product without saying anything about it is off-putting to me [...] Fair enough. Fuzzbuzz isn’t quite ready for public access yet, so that’s why we’re a bit vague here, but the intention was not to advertise our product (and is why we only wrote a couple paragraphs at the bottom). We were just excited to write a post about fuzz testing, and figured anyone who’s interest was really piqued could get in touch. We hope to expand this post and use this as an educational resource long-term. [0]: https://en.wikipedia.org/wiki/Differential_testing [1]: https://guidovranken.com/2019/05/14/differential-fuzzing-of-... [2]: https://github.com/sigp/beacon-fuzz [3]: https://hypothesis.readthedocs.io/en/latest/stateful.html?hi... |