[pfdietz]

For more than 20 years I've been doing automatic test input reduction as part of testing Common Lisp compilers. The reduction is on randomly generated inputs, but they are structured in such a way that reduction always gives a valid program that should (in the absence of compiler errors) not signal an error.

It's a tremendously economical way to test compilers. For a modest and finite investment in testing infrastructure I get an unlimited number of tests. Over the years I've run many billions of test inputs on various Common Lisp implementations, although I'm mostly focusing on sbcl these days. When a bug is found the input quickly reduces to a something small that usually immediately tells the developers where the problem is (usually but not always something introduced recently.)

I also have a testing harness that cobbles together usually erroneous Lisp code and sees if the compiler blows up (the sbcl compiler as designed must never throw an error condition even on erroneous input.) This exploits a corpus of public Common Lisp code, combining and mutating the code in various ways.