And thanks to AI, we could generate extremely convincing reams of code whose only purpose is to be fake unit tested. Amazing. I sincerely hope I never need to use this nuclear weapon.
Or better yet: effectively fake unit tests. It is almost never the case that tests written by AI detect actual issues. At most they detect that has changed.
Yeah I’ve been thinking that LLM unit tests are basically snapshot tests. Just sorta ossify things in place. If they break, you just ¯\_(ツ)_/¯ and have the LLM fix them. It’s like they were never there!
So it’s just like the olden days of everybody ignoring tests, but we give anthropic a ton of cash