[wiseowise]

> You assert that it writes better code than the average software developer?

Absolutely. It contains a lot, if not majority, of all the code available at our hands right now and can reason, whatever it means for LLMs to reason anyway, about it. It absolutely demolishes average software developer and it’s not even close.

> To achieve code coverage it's enough to CALL the code, it doesn't tell you anything about the correctness of the tests: they could all end with a return true, and a code coverage tool would be perfectly happy.

> So, yes, if you don't carefully check the test suite that the agent writes, it might well be worthless (or simply much less useful than you assume it to be, more realistically).

That’s like saying that if you don’t check every line your coworker writes it becomes worthless.

[g-b-r]

You seem to have a pretty big misunderstanding of LLMs.

> > So, yes, if you don't carefully check the test suite that the agent writes, it might well be worthless (or simply much less useful than you assume it to be, more realistically).

> That’s like saying that if you don’t check every line your coworker writes it becomes worthless

A coworker is (supposedly) a *competent person*, placing some trust on that is not stupid.

You'd usually want to have a review of everything everyone does, but even if you don't do it, a reasonably competent and honest developer is never going to be as misleading as LLMs can be.

Furthermore, in traditional development, even without code reviews you will look at the test suite every once and then (or rather, always, when you touch the code that it tests).