[skydhash]

> Meanwhile, AI is happy to write tests, do red-green TDD cycles, refactor them, prune them, update them, justify and defend them. It will even incidentally write tests for the most aloof vibe-coder by accident because they didn't specify otherwise.

I read some AI generated tests and while it looks visually impressive, ultimately it wasn’t doing anything valuable? Why? because of all the mocks and scenarios that didn’t matter. And on top of that, tests are additional code to maintain.

These days, I don’t even bother with unit testing. They are a maintenance burden. I focus on integration test (whole modules) and if I have the time, on a harness to do e2e testing.

[icedchai]

Many humans do the same thing. They write tests that mock so much of the actual code, the "test" is not testing much at all, except perhaps the developer's ability to basically turn the code inside out. It's often just a large volume of crap that has to be maintained (or eventually deleted.)

I agree integration tests are best, with some e2e testing for common scenarios.

I worked at a place that required unit tests for every new method or function. Arguments like "this other (integration) test already covers that. why do I have to add another test?" wouldn't fly. PR reviews would often degrade into arguments about testing and how all database access needed to be mocked...

[mega_dean]

> I read some AI generated tests and while it looks visually impressive, ultimately it wasn’t doing anything valuable

I just saw this comment yesterday about one of the tests from Bun’s rust rewrite: https://news.ycombinator.com/item?id=48314311 It reads in the raw source code and uses a regex to assert that “unsafe” is used.

> These days, I don’t even bother with unit testing. They are a maintenance burden

I’ve come to the same conclusion, but that’s only because I’m working on solo projects. I think they are probably worth it with multiple devs on the same project.

[majormajor]

IMO with multiple devs the focus on integration (module-level) instead of unit (lower-level/helper-function-level) is even more important than on single-dev projects.

[ThunderSizzle]

My instructions and steering is to force the LLM agent to focus on mocking only system boundaries and outside in unit testing. I've found these two make verbose but good tests that actually prove behavior pretty well.

But that was my testing strategy already, but writing one of those tests could take legitimately hours with how many columns and crazy rules our area has.