Hacker News new | ask | show | jobs
by thyrsus 3593 days ago
My understanding is that fuzz testing uses pseudo-random variation of the seed code; given a different seed to the PRNG, how common is it for the same fuzz test to identify different flaws?
1 comments

In my experience very rare. It's so rare I'd say it basically doesn't happen.

You'll get some fluctuation, a bug may come up half an hour earlier or not. But results tend to be pretty reproducible. If you find a bug with a specific test and tool in x hours then the next time you try for at least x+1 hours you'll find it again.

I think that's right for just picking a different PRNG seed. When you start looking at modifying the search heuristics, mutation operators, or other parts of the "strategy", you definitely start finding different bugs, though.
Yes, absolutely. Different strategies can lead to vastly different results, often it's subtle things.

Good example: There was a bug I found in openssl that Libfuzzer was unable to find. The Libfuzzer developer was quite interested in this and has now adopted new mutation strategies: https://github.com/google/sanitizers/issues/710