[exe34]

> Checking is usually faster than writing from scratch

Famous last words. Checking trivial code for trivial bugs, yes. In science, you can have very subtle bugs that bias your results in ways that aren't obvious for a while until suddenly you find yourself retracting papers.

I've used LLMs to write tedious code (that should probably have been easier if the right API had been thought through), but when it comes to the important stuff, I'll probably always write an obviously correct version first and then let the LLM try to make a faster/more capable version, that I can check against the correct version.

[dcrazy]

Humans are notoriously bad at checking for meticulous detail—hence why copyediting was a specialized role. We’ve seen what’s happened since the newspapers eliminated it for cost savings.

I only used an LLM for the first time recently, to rewrite a YouTube transcript into a recipe. It was excellent at the overall restructuring, but it made a crucial and subtle mistake. The recipe called for dividing 150g of sugar, adding 30g of cornstarch to one half, and blanching eggs in that mixture. ChatGPT rewrote it so that you blanched the eggs in the other half, without the cornstarch. This left me with a boiling custard that wasn’t setting up.

I did confirm that the YouTube transcript explicitly said to use the sugar and cornstarch mixture. But I didn’t do a side by side comparison because the whole reason for doing the rewrite is that transcripts are difficult to read!

[libraryofbabel]

> I'll probably always write an obviously correct version first

I’m not usually so confident in my own infallibility, so I prefer to think of it as “I might get this wrong, the LLM might get this wrong, our failure modes are probably not very correlated, so the best thing is for us both to do it and compare.”

Agree it is always better for the human engineer to try writing the critical code first, since they are susceptible to being biased by seeing the LLM’s attempt. Whereas you can easily hide your solution from the LLM.

[exe34]

> I’m not usually so confident in my own infallibility

I'll concede that point, but let me put it differently: when I write the "obviously correct" version, what I mean is that I can explain what every line of code is supposed to be doing. I can say what the expected result would be, at each step and test it.

When the LLM writes an inscrutable version for me, I might be able to test the final result, but I'll never know if there are corner cases (or not even that) in between where it will go spectacularly wrong. If I have to untangle what it's written to get that kind of confidence, then it's going to take me much longer. Reading code, especially somebody else's, nevermind an alien's code, is just so much harder for me.