[KronisLV]

Vouching for this comment because my friend confided in me a week ago that her manager also does this and is like “oh yeah, here’s 80% done, you just do the rest so we can ship it” when a large part of it is slop that needs to be rewritten, due to not enough guidance and pushback during generation.

[eCa]

That’s when you ask it to write tests to a good coverage, and then have it reimplement everything with the tests still passing…

[camdenreslink]

Writing tests against a bad implementation usually doesn't work well. In this scenario I would have an LLM look at the changes in the branch and try to create a markdown document of the changes, why it thinks they were made, etc. and then review that doc with the manager and do a new implementation from scratch after aligning.

[KronisLV]

Unless the tests are written against logic that is in of itself subtly wrong and even the structure of code and what methods there are is wrong - so even your unit tests would have to be rewritten because the units are structured badly.

It’s a valid direction to look in, it just doesn’t address the root issue of throwing slop across the wall and also having unrealistic expectations due to not knowing any better.

[skydhash]

Yep. It’s very healthy to be suspicious of code. Any code. Whether generated or not. That’s where the bugs are.

If there’s one thing that’s disturbing with AI proponent is how trusting they are of code. One change in the business domain and most of the code may have turn from useful to actively harmful. Which you have to rewrite. Good luck doing that well if you’re not really familiar with the code.

[throwaway173738]

Be careful with this because the LLM will just change the tests on you to get them to pass.