[outsidetheparty]

The few times I've tried using ChatGPT or another LLM as a coding assist, the "confidently wrong answer that looks correct" was the entirety of my experience. (Mostly the failure mode was mixing up incompatible instructions from various versions of the framework or toolchain: even if I specify a version number it'll still often want to use syntax or functions that don't exist in that version.) I did not find it to be a time saver.

[padolsey]

Hmm fair. It's strange that our experiences are so different. Can I ask what types of problems you ask it to solve? FWIW I've had to take quite a lot of time figuring out how to talk to it in a way that gets good results.

[outsidetheparty]

The one where I struggled the longest was trying to put together the right webpack configuration to generate multiple static files based on input in markdown format. It kept switching which plugins it wanted me to use, or mixed up functions from conflicting plugins, and often mixed up syntax from different versions of webpack itself.

I finally gave up on that one when it got caught in a loop somehow where it apologized for giving me the wrong line of code for an import, gave an obviously wrong explanation for why it didn't work, then "corrected" it to the exact same line of code.

Another attempt I was asking it to compare different ways for measuring the amount of difference between data trees -- it did give me the names of a couple of different algorithms, and very wordy, plausible-looking descriptions of how each of them worked... neither of which was terribly helpful, because they both boiled down to "recursively examine the tree and tally up the differences."

Asked for an implementation example, it gave me code that expected input arrays of pre-calculated edit costs, and suggested I write my own function to convert the tree data into that array format.

So that one was extra weird, in that it wasn't wrong, just unhelpful, like here I'll do the easy part for you and leave the thing you asked about as an exercise for the reader.

I dunno, maybe with practice I could learn how to drag it towards helpfulness, but for now RTFM still seems easier.