[hallway_monitor]

Exactly.... -> Unit tests. Integration tests. UI tests. This is how code should be verified no matter the author. Just today I told my team we should not be reading every line of LLM code. Understand the pattern. Read the interesting / complex parts. Read the tests.

[GrinningFool]

But unit and integration tests generally only catch the things you can think of. That leaves a lot of unexplored space in which things can go wrong.

Separately, but related - if you offload writing of the tests and writing of the code, how does anybody know what they have other than green tests and coverage numbers?

[dntrkv]

I have been seeing this problem building over the last year. LLM generated logic being tested by massive LLM generated tests.

Everyone just goes overboard with the tests since you can easily just tell the LLM to expand on the suite. So you end up with a massive test suite that looks very thorough and is less likely to be scrutinized.