Hacker News new | ask | show | jobs
by hannob 3593 days ago
In my experience very rare. It's so rare I'd say it basically doesn't happen.

You'll get some fluctuation, a bug may come up half an hour earlier or not. But results tend to be pretty reproducible. If you find a bug with a specific test and tool in x hours then the next time you try for at least x+1 hours you'll find it again.
1 comments
moyix 3593 days ago
I think that's right for just picking a different PRNG seed. When you start looking at modifying the search heuristics, mutation operators, or other parts of the "strategy", you definitely start finding different bugs, though.
link
hannob 3593 days ago
Yes, absolutely. Different strategies can lead to vastly different results, often it's subtle things.

Good example: There was a bug I found in openssl that Libfuzzer was unable to find. The Libfuzzer developer was quite interested in this and has now adopted new mutation strategies: https://github.com/google/sanitizers/issues/710

link