[ianthehenry]

Mutation testing is where you make changes to your code's logic as a way to ensure that you have good test coverage for that bit of logic. Mutation testing is helpful with any kind of tests -- assertions, snapshots, whatever -- as a way to check that your tests are actually able to catch interesting bugs, but the idea is orthogonal to this workflow.

[trolan]

Have you found any added benefit in your approach using mutation testing?

Really interesting stuff I'm going to play with personally and in school I think. Thank you.

[ianthehenry]

Hmm, I like mutation testing, and have found it very useful, but I don't know if it feels any different with this technique as it does with traditional assertion tests. Perhaps one difference is that, when you find a hole in your test coverage, it's easier to fill it in when tests are easier to write.

One thing that is a little different is that this type of testing makes it easier to write "counterfactual" tests, which are sort of similar: you observe the behavior of a function with input A, and then you make an interesting tweak to the input to get input B, and you can see how the output changes (you can even print the diff between the two outputs in your test). This is useful for ensuring that a test is actually testing exactly the thing that you want it to test -- if you have a test that "X fails validation when Y is Z," you can ensure that the only difference between a failing and a passing X is the value of Y, whereas if you only test that "X with Y=Z fails," you won't be totally confident that the failure is due to Y or due to some other newly introduced E. I think that the ability to print diffs between inputs and outputs makes this kind of REPL test a lot easier to read and write in this case.