Hacker News new | ask | show | jobs
by zhd 1800 days ago
These are basically the problems that https://hypofuzz.com is designed to solve.

- You write standard property-based tests with Hypothesis, which fit into a traditional unit-test workflow (run with pytest, or unittest, or whatever)

- then HypoFuzz runs indefinitely, using feedback-guided fuzzing to find bugs that random generation is too slow for

- restarting the search on each new commit (or daily, etc) begins by replaying every distinguishable input found earlier so you don't have to start from scratch. There is _some_ loss of context, but that's inherent in testing different code.

Plus other nice stuff like allocating more compute time to tests which are finding new behaviour faster, for maximum efficiency. The roadmap includes exploiting SAT solvers to find very very rare behaviour, git history to focus on recently changed code, statistical risk analysis ("what's the expected time to next bug"), and also a budget-mode ("my compute costs $. 08/hr, stop when you expect a bug to cost $100 or more to find"). It's a good time to be working on testing!