[vjvjvjvjghv]

It always amazes me that people don’t even check the output of the AI.

I am at the moment going through a codebase where the dev told me that he had AI write the tests and that the project is finished. Turns out that some tests are totally nonsensical and a lot of the real world edge conditions aren’t tested at all.

I am all for AI but you can’t just blindly let it do its thing and think you are fine.

[welshwelsh]

Test cases being nonsensical and not testing any real world cases was extremely common even before AI, in my experience.

When teams rely on metrics like code and branch coverage, developers will write tests that only serve to satisfy those metrics. I've seen tests that end with "assert true == true"; they don't test anything and only exist to "cover" a branch.

The situation has improved vastly because of AI. AI agents frequently write good tests that have a real chance of actually catching bugs and improving the reliability of the application.

[icedchai]

This happens all the time. I looked at a PR, many tests were broken, told the guy he needed to at least fix the previously working tests, got tons of slop like changing assert 200 to 500.

[vouaobrasil]

It's human nature to use a probabilistic strategy of cheating until one cheat works. We will never eradicate that, and the careful person that checks the output of AI will always be a minority.