Hacker News new | ask | show | jobs
by Hnrobert42 2 days ago
I read the whole article, and I am still confused. I get that test case reducers find the smallest error causing string. I don't understand why that is particularly valuable.

Also, do test case reducers work on integers or other numbers? What about reducing some other complexity? Is this for developing unit tests or just debugging?

3 comments

A reduced test case means you run less code to process the test case, which means your breakpoints trigger less frequently (and the remaining breakpoint triggers are more likely to be relevant to the actual bug). It also means all your debugging steps are likely to run faster and produce less data to sort through. Your log files will be shorter and easier to read/grep, etc.

Imagine being handed a sheet of 10 equations and being told "1 of these equations is wrong." Now imagine that someone came in and erased 8 of the correct equations - they just saved you a bunch of time.

Making the smallest test case that reproduces some bug is hugely valuable when debugging complex systems, especially if you have a wierd heisenbug that is hard to manifest reliably. Having a small reproducible case massively narrows the scope of the search for the bug.

Similarly, narrowing test cases to the smallest case that reproduces a particular behaviour so you're only actually testing a very targeted thing will make the test suite faster and also make it easier to fix tests which break because they exercise a very narrow path.

Reduced test cases make it way easier to figure out what the actual bug is. E.g. a real world example is when I tested a "slugify" function with hypothesis. It almost immediately spat out this failing test case:

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaẞ

Boy I sure do wonder what character could possibly be causing issues there. whereas without shrinking it might instead spit out something like

ЁЂЃЄЅІЇЈЉЊЋЌЍЎЏАБВГДẞЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмноп

Which makes finding the offending character a lot harder.