[mlmonkey]

Writing unit tests used to be the bane of my existence. I used to hate them. Often times, the LoC for unit tests was 3X the LoC of the actual code.

But not any more! Now I point the LLM to the code and order it to write unit tests, covering all edge cases, etc. I'd rather spend 3 hours arguing with the LLM than writing unit tests! :-D

[dkn]

I am curious in your experience how often the LLM must also update the tests. I find that if LLMs write tests after the implementation exists, they are either extremely brittle because they are coupled to the implementation, or they cover little of value because they mock everything to the point of testing nothing.

[mplanchard]

I have found a decent trick to be to write a parameterized test with e.g. a `cases` array that tests a function how you want it tested. Then ask the LLM to fill out more cases. It’s not perfect, but results in much less brittleness since you’ve already defined the specifics of what gets tested and what doesn’t.

[dcastm]

Same for me. I actively ask the LLM to write as few tests as possible. Otherwise you end up redundant and low value ttests.

[dkn]

Yep, and wasted token spend on an ongoing basis.

I instruct the LLM to follow TDD practices in certain areas, but otherwise prioritize integration style tests at the edges.

[kovek]

I heard people say this before. I'm wondering, how do you instruct the LLM to generate the tests? Do you tell it the scenarios that would be covered, or do you just tell it to write tests for the code?

[zerr]

Some companies (e.g. Microsoft) used to have "Software Engineers in Test" who's job was writing such tests all day long, so that those developers who were developing features wouldn't waste time on it.