Hacker News new | ask | show | jobs
by frje1400 656 days ago
LLM output can't really be trusted so I need to "proof read" it and convince myself that it is correct. In the language I use every day and have a high degree of fluency, it's faster for me to simply write what's in my head than to proof read unknown code. So how can LLMs make me more productive in actual programming?

I use an LLM to generate ideas, to rubber duck, to get a lead on unknowns, and to generate boilerplate occasionally. So I do everything except replace the coding part because that's what requires the most precision, and LLMs are bad at precision. And yet, people claim massive productivity gains in specifically coding. What am I missing?

4 comments

It helps a lot if you use a strongly typed language with a strict compiler and get the LLM to write plenty of tests. Then you need to understand that the tests are logically correct. The LLM is also good at documenting the functions, so you can review that matches your intentions and the code as well. Also the LLM will pick functions from your own source code library to compose new programs for you. So the reuse of your own well tested code should increase confidence.
> So how can LLMs make me more productive in actual programming?

Suppose you suddenly are required to write a VBA macro for Excel for your job. It's a one off task - not something you'll do repeatedly. Do you prefer learning VBA for Excel and crafting a solution or asking the LLM and verifying its solution by looking at the docs?

Hint: If you use the macro recorder in Excel and inspect the code you are closer to the LLM end of the spectrum.

This is basically the same conclusion I’ve come to, after 18ish months of almost daily use.
Also, the errors that an LLM makes are not errors that a typical human would make. This makes reviewing their code particularly challenging.
I find the errors they make are often comically "human", like forgetting a premise.