[ath3nd]

> LLM for a complex piece of software far beyond what you have time for, you see the benefits.

Are LLMs the new Agile/Scrum?

"Once you really learn Scrum, it will solve all world problems and clean your house? Your experience is totally different? Skill issue. Try again."

I get your position and don't want to sound dismissive either, however I want to point out that in the only recent study actually trying to measure the productivity gains of LLMs it was observed that there is an actual 19% reduction of gains for experienced developers when using an LLM.

https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...

I asked an LLM to tell me why it "thinks" you observe an increase of productivity while studies show that for experienced developers it's a decrease, and it came up with the following "ideas":

"LLMs Fill Gaps in Knowledge Instantly. Junior developers often face friction from:

- Unfamiliar syntax

- Unclear documentation

- Uncertainty about best practices"

Again, I don't want to sound dismissive, but have you considered that instead of people not seeing the gains you are talking about due to a skills issue with how to fine prompt LLMs, that it's you seeing gains you wouldn't otherwise had you been more skillful?

[calrain]

If knowledge and experience isn't an issue, then LLMs will benefit the programmer less in that space, but are still useful for doing mundane activities you avoid doing, like pivoting an early idea about an API pathing strategy, and have the LLM do the test case pivot for you.

If knowledge and experience in the language is an issue, then LLMs have increased value as they can teach you language notation as well as do the mundane stuff.

If understanding good programming architecture / patterns is an issue, then you have to be more careful with the LLM as you are listening to advice from something that doesn't understand what you really want.

If understanding how to guide an LLM is an issue, then you have to work, test, and design ways of building guidelines and practices that get the outcomes you want.

Using LLMs to code isn't some cheat-code to success, but it does help greatly with the mundane parts of code if you know how to program, and program well.

How much of a large project is truly innovation? Almost every application has boilerplate code wrapped around it, error handling, CRUD endpoints, Web UI flows, all stuff you have to do and not really the fun stuff at the core of your project.

This is where I find LLMs shine, they help you burn through the boring stuff so you can focus more on what really delivers value.

[booleandilemma]

This corroborates my observation that the people I've seen most excited about LLMs are management types who know how to code but not really. They love to talk about how productive LLMs make them. I think the alleged increased productivity is just in their heads.

[ath3nd]

There are always gonna be those business that claim that LLMs/Scrum/NFTs/Crypto/web3 is the next coming of Jesus, and that's normal, their survival depends on that. There are also gonna be developers swearing by XYZ, either because it genuinely worked for them (causation is not correlation) or because they don't want to appear clueless in an industry that's vibe based and tells you that if you don't succeed using shiny tool XYZ, you are doing it wrong.

The same way it was modern in some companies to hire a dedicated Scrum master for a full time position within a team, I already can imagine companies having job openings for an "Expert LLM-prompter" (or, to make it sound more legit: "Staff LLM-prompt engineer"). Take it from the funny side: the companies obsessed about productivity and efficiency will most likely be staffed with 100s of scrum masters and 100s of prompt engineers and probably one or two guys that actually do the real work. That's kind of hilarious, you gotta admit.

The emperor has had no clothes for quite some time already, but vibes do be vibing, that's the society we live in. Don't worry, you are not alone in finding the hype hard to bear. I don't think LLMs will become irrelevant as fast or as much like crypto/web3/nfts and Meta's metaverse did, but to me the amount of shouting and clothes tearing for what's essentially a fancy autocomplete (fight me on this) is just such a turn off.

[dnh44]

I think that in that study all but one of the devs had hardly any experience with using AI tools.

[ath3nd]

> devs had hardly any experience with using AI tools.

I love it, here come the "you are using it wrong" arguments!

I thought these tools are so great, so awesome, that even without much experience they will help you? Or are they not so great and you actually gotta spend a considerable amount of time of learning them to see some return? Which one is it:

- are LLMs groundbreaking and democratizing development making it so much easier (which doesn't correspond to the results of the study)

- or do they need months of practice to give a modest return (or loss) of productivity back?

"You are using it wrong" is the cheapest cop-out when somebody questions the productivity benefits of LLMs. I'd like the LLM fanbase community to come up with better arguments (or ask their little assistant for assistance).

https://news.ycombinator.com/item?id=44854649

[dnh44]

>I thought these tools are so great, so awesome, that even without much experience they will help you?

I never made that claim, I don't think it's generally true, but I do think it can be true for some people.

Regarding the study, I pointed out why I thought it was limited in its scope. If someone did a study on the potential productivity gains of using emacs and they based the study on people who had only used it for a week it wouldn't be a particularly useful study. A Piano is not a useful tool for making music for someone who has only used it for a week either.

I do have a customer with no prior programming experience that has created custom software for his own small manufacturing business automating tedious tasks. I think that's pretty amazing and so does he.