[jgalt212]

These "solutions" place a lot of faith in a "complete" set of test cases. I'm not saying don't do this, but I'd feel more comfortable doing this plus hand-generating a bunch of property tests. And then generating code until all pass. Even better, maybe Claude can generate some / most of the property tests by reading the standard test suite.

[grogers]

Well they also shadowed production traffic and fixed some bugs that were causing mismatching results. Not saying that stuff can't still slip through, but it's a good way to evaluate it against real data in a way you can't from just test cases alone

[jgalt212]

parallel execution that auto-generates test cases from exceptions is very slick. That being said, you still need humans in the loop as sometimes the oracle is not THE oracle.