Some feedback going through the docs. For each of the languages you demonstrate a "BrokenMethod" but I don't understand what's broken about it. e.g.: func BrokenMethod(Data string) bool {
return len(Data) >= 3 &&
Data[0] == 'F' &&
Data[1] == 'U' &&
Data[2] == 'Z' &&
Data[3] == 'Z'
}
What's broken about this? It returns true for strings that start with "FUZZ", false otherwise, does it not? Python example: def BrokenMethod(strInput):
if len(strInput) >= 2:
return strInput[0] == 'F' and strInput[1] == 'U'
Other than not being idiomatic I don't see what's wrong with this method.Next, it's not clear to me how you indicate success/failure of a test. Is success just any program than exits 0 and failure any program that exits non-zero? That would be my guess but the docs don't say. Typo: https://github.com/fuzzbuzz/docs/pull/3 This page is missing a link to the find-your-first-bug-in-Python example: https://github.com/fuzzbuzz/docs/blob/master/getting-started... The docs site loads slowly for me on an older iPad, and there's even a slight delay on a recent Macbook. Looks like it's maybe a font loading issue? (Oh, it's gitbook. How awful. I guess there's nothing you can do about that other than use a different doc provider.) |
1) The BrokenMethods are simple examples of programs that crash on buffer overflows/index out of range errors. If you were to pass "FUZ" into the Go method, it would check Data[3], thus causing a panic since there are only 3 elements in the string.
ninja edit: that python method in your comment IS a valid method with no error - a bit of a brain fart on my end when writing out the docs. It's been changed :)
2) In general a failure is any non-zero exit. We do this to be flexible in the way you report bugs. For C/C++ and Python this is usually with assertions, and in Go you can achieve something similar with:
We also have other checkers or "sanitizers" that run with your code to look for certain bugs. For C and C++ code we support tools like Address Sanitizer, which report memory bugs like Heap Buffer Overflows and UAFs, and for Golang you can choose to fuzz your code with a race condition checker. These are just some of the examples of more advanced fuzzing methods we support, and we'll be making nicer tutorials/screencasts to showcase those over the coming week.3) Thanks for the fixes - much appreciated. And yeah, we know GitBook is pretty slow, and we're in the process of moving to another docs provider.
If you've got any more questions please let me know!