Hacker News new | ask | show | jobs
by adkadskhj 1909 days ago
I'm quite new to property testing, first introduced recently via a Rust property testing framework proptest[0]. So far i've had the feeling that property testing frameworks need to include a way to rationalize complexity, as their assumptions can easily fall short as you illustrated.

Eg the simplest example might be an application where you input an integer, but a smaller int actually drives up the complexity. This idea gets more complex when we consider a list of ints, where a larger list and larger numbers are simpler. Etcetc.

It would be neat if a proptest framework supported (and maybe they do, again, i'm a novice here) a way to rank complexity. Eg a simple function which gives two input failures and you can choose which is the simpler.

Another really neat way to do that might be to actually compute the path complexity as the program runs based on the count of CPU instructions or something. This wouldn't always be a parallel, but would often be the right default i imagine.

Either way in my limited proptest experience, i've found it a bit difficult to design the tests to test what you actually want, but very helpful once you establish that.

[0]: https://crates.io/crates/proptest