[fragmede]

I’m exhausted by these types of posts.

They never include concrete details on what they're trying to do. what languages they're using, what frameworks, which LLM. They occasional state which tool, but then don't go into detail how they're using it. There's never any links to chats/sessions showing the prompts they're giving it and the answers they're finding so unacceptable.

Imagine if you got bug reports from customers with that little detail.

Actual in-depth details would go a long way to debugging why people are reporting such different experiences.

It takes a back and forth exchange with the LLM for it to make progress. Expertise in using an LLM is not just knowing what to prompt it, but more importantly, when to stop, fix the code yourself, and keep going. Without throwing baby out with the bathwater just because you still had to do something by hand, where the baby is "using an LLM. in the first place".

If I had to guess though, I think that's where people differ. Just like with every skill there's a beginners plateau and you hit a wall and have to push through (fatigue/boredom/disillusionment/etc). If the way you're using the LLM means you haven't gotten a hallucination by then, and you've seen how wildly more productive and how it's able to take away some of the bullshit in programming; if no bad stuff has hit the wall and you take to it like a fish in water, you can push through some of the dumber errors it makes.

If, however, you are doing something esoteric (aka not using JavaScript/python) and are met with hallucinations and scrutinize every line of code it produces, even going into it with an open mind, it's easier to give up and just stop there. That may not even be the wrong thing to do! Different programmers deliver value in different ways. You don't want Gilfoyle when you need Richard Hendriks, or vice versa, a company needs both of them.

So: show us the non-functional wall on GitHub the LLM built, or even just name the language used and the library it hallucinated.

But again, getting perfect code out of the LLM is a non-goal, don't get distracted by it. LLM-assisted or not, you get graded on value derived from code that actually gets committed and sent for review and put into production. So if the LLM is being dumb, go read and fix the code, give it your fixed code, and move on with your life, or at least into the next TODO/ticket.

[wruza]

No one includes complete detail at saying it’s useful and life-changing too, so that’s fair. It might turn out that what works for those for whom it works is trivial “code” not worth the ssd blocks it occupies. This is actually my current theory, cause LLMs (all of them, yes we tried all of them) are capable of what I tend to not think about as programming but as industry nonsense which should have been automated/abstracted/libraried away ages ago.

Maybe show us the successful code it built and we’ll see what type it is, cause recording failures is only useful in hindsight. I have no logs of lenghty struggling with llm stupidity.

getting perfect code out of the LLM is a non-goal

It stops being a goal after just a few tries, naturally. The problem is usually not that it isn’t perfect, the problem is it doesn’t understand the problem at all and tends to some resembling mediocrity instead. You can’t just fix it and move on.

[handzhiev]

"No one includes complete detail at saying it’s useful and life-changing too"

There are at least 3 posts in this very discussion sharing details and githup repos with code written mostly by LLM.

[wruza]

I see a weather.com class, a parser and a react gui boilerplate boilerplate tsx folder. All three are textbook and trivial areas. We are only missing an ad hoc crud orm here.

In my opinion that is not code, it’s not a business logic. What is presented is (not to offend anyone, I look at code not people) useless github-code carcasses that it contains in abundance. Real code solves problems, this code solves nothing, it just exists. A parser, a ui, an http query - it’s a boilerplate boilerplate boilerplate. You aren’t coding at writing it. It’s “my arduino is blinking leds” level of programming.

I think that’s the difference in our perception. I won’t share my current code purely for technical reasons, but for an overview, it fuzzy-detects elements on virtual displays and plays simple games with dynamic objects on screen, behaving completely human input-wise, all based on a complex statistical schedule. It uses a stack of tech and ideas that llms fails at miserably. Llms are completely useless at anything in there, because there’s basically no boilerplate and no “prior art”. I probably could offload around 15% to an llm, but through the pain of explaining what it’s supposed to assist with.

Maybe it’s me, but I think that most of the jobs that involve trivial things like “show fields with status” or “read/write a string format” are not programming jobs, but an artefact of a stupid industry that created them out of mud-level baseline it allowed to persist. These should have been removed long ago regardless of AI. People just had way too much money (for a while) to paycheck all that nonsense.

Edit: I mean not just removed, but replaced with instruments to free these jobs from existing. AI is an utterly sarcastic answer to this problem, as it automates and creates more of that absurdity rather than less.