[bluefirebrand]

The test was "can I get it to generate this while spending less effort than it would take me to just write it" and it failed miserably. And this was a super low effort, small boilerplate problem to solve. This is the sort of problem it has to solve to be remotely useful

If it cannot do that, then why is anyone saying it is a productivity booster?

[knowaveragejoe]

My response could only possibly be that I haven't had that issue. I've asked for relatively complex changes to codebases(mainly python), and had very little in the way of trouble.

[imiric]

... That you're aware of.

The more code you ask it to generate, the higher the chances that it will introduce an issue. Even if the code compiles, subtle bugs can easily creep in. Not unlike a human programmer, you might say, but a human programmer wouldn't hallucinate APIs. LLMs make entirely unique types of errors, and do so confidently.

You really need to carefully review every single line, which sometimes takes more effort than just writing it yourself. I would be particularly wary of generated code for a dynamic language like Python.