[gashad]

This reminds me of themes I recently saw in [Harness Engineering is not Enough: Why Software Factories Fail](https://www.youtube.com/watch?v=Ib5GBkD555M) (Warning: the last 3 slides seem like an advertisement). One thing I liked is how Dex has a little graphic he glossed over showing software development is

- 25% planning & aligning with other teams

- 25% coding

- 25% testing/verifying

- 25% code review/rework

One argument was that agentic coding speeds up that coding part a bunch. So maybe there's 2x speedup in coding. But that's only a small speedup in the totality of everything software engineers do.

[altern8]

AI is great because my job is a lot easier, but it produces some extremely crappy code that would've NEVER passed a code review 1 year ago.

You could argue that it speeds up development by 5X or more, but then it slows down testing/verifying, code review, and in many cases it makes it impossible to review/rework by hand.

I just do what I'm told at work but even though I'm sure I'm in the minority I'm extremely skeptical that LLMs can produce any good-quality code.

Looks to me like we've just lowered the bar--by a lot--and stopped looking at the code that goes out. Every time I look into how Claude implemented something it's completely insane, with no way to refactor it or maintain it in the future.

[serial_dev]

> it produces some extremely crappy code that would've NEVER passed a code review 1 year ago

The 10x speed up comes from the fact that we have all given up on properly reviewing each others code and we just say “meh, it will be fine, lgtm”

[QwenGlazer9000]

This. The bar has never been lower.

[dools]

Code review has always been pointless

[SkyPuncher]

I realize this like a decade ago. Security is basically the only thing I care about in code review.

I’ve come to realize that I don’t actually care what your code looks like. I’m going to need to learn it from scratch every time I use it. It’s going to look weird and foreign to me no matter what. It’s not worth my time nit-picking things that simply do not matter.

[zhoujianfu]

I’m glad somebody said it!

[altern8]

Yes, exactly my point.

BUT, to me if we hadn't lowered the bar MASSIVELY, it would NOT be fine.

I've seen people at my work that will push or even deploy features that don't do what their main purpose is and they just dismiss the issue saying it's a bug. As an example, a button that's supposed to open a modal window if you click on it. If you go test it and the modal doesn't open, they'll say "oh, there's a bug".

You developed or even deployed something that is wrong and completely useless. Just one year ago, this would be unheard of and you would've gotten a really hard time for it. Now it's just a bug and part of how we're doing things.

Like I said, I just do what I'm told at work but to me this is just wild. There's no way it won't come back to bite us.

[elcritch]

I read this a lot, do you have examples of this? Last year the code produced by LLMs was pretty sloppy, but recent SOTA models seems pretty good to me. I'm curious if I'm missing something or it's folks using different models or difficulty levels.

[altern8]

From my experience, a snippet of code will look good.

But LLMs seem to have a hard time getting the big picture and reusing code that is already implemented and ALMOST does what you want vs. rewriting everything from scratch.

AI code is extremely difficult to read and follow. It's littered with hundreds of lines of comments and notes, often referring to other notes in other parts of the codebase, and often extremely out-of-date.

Claude seems to me to still focus on getting things done vs. doing it properly, so from my experience it would pretty often mess up your whole codebase just to be able to finish its task, vs. stopping and rethinking the approach.

After a few passes of that, with duplicate code with no underlying coherent vision, hundred of thousands of lines of documentation written as walls of text in markdown and weird coding decisions, your codebase is impossible to work out for a human.

I use Claude Opus 5 at high to plan, Claude Opus 5 at low to execute the plan.

I make it build things in small changes, I give very specific instructions about the architecture, and I make sure to point out existing functionality that can be used instead of writing something from scratch. But it's a lost battle because Claude can't learn and it's basically a black box.

It pretty shitty, but that's what they pay me for so I just do it.

[jaggederest]

> It's littered with hundreds of lines of comments and notes, often referring to other notes in other parts of the codebase, and often extremely out-of-date.

Adjustable by tooling and prompts - my favorite is a ban on comments >2 lines, a tagged index system for effectively in-repo RAG with short documents on each tagged topic like INFRA-DATABASE-SETUP.md - enforce with tooling that the tag matches the doc and vice versa. Enforce that any PR that has a tag change in it has the relevant doc changes.

> Claude seems to me to still focus on getting things done vs. doing it properly, so from my experience it would pretty often mess up your whole codebase just to be able to finish its task, vs. stopping and rethinking the approach.

Tell it your values - "I value correctness over getting things done, and getting things done properly over speed" solves many of those problems. I've actually spent a portion of today rejiggering my values documents because the models have improved enough I no longer need to be quite so careful about scoping.

I was having those same problems, but particularly using Fable / Sol to do the meta-work has largely eliminated them over the last ~3-4 months. Codebase is now a lean mean token-conserving machine.

[elcritch]

Thanks! Many of your issues sound more agent / LLM specific than LLMs in general. At least compared to my experiences.

It also aligns with what I saw from Claude & Claude Code when I used it last year for a while. Now I use Codex and don't see (nearly) as much of that sort of behavior.

> But LLMs seem to have a hard time getting the big picture and reusing code that is already implemented and ALMOST does what you want vs. rewriting everything from scratch.

Yeah that's probably the weakest point of LLMs still. GPT-5.6 Sol got much better at this for me. I still usually end up doing 2-3 iterations of prompts and exploration to clean up various things but usually it's pretty light work now.

> hundred of thousands of lines of documentation written as walls of text in markdown and weird coding decisions, your codebase is impossible to work out for a human.

Okay Claude definitely seems to have an issue there. From what I've seen from coworkers using Claude it generates reams of endless docs. I resist the urge to `rm docs/planning*.md`! I don't think they get how bad Claude is at that.

A month back I tried the latest DeekSeek and it made reams of text back and forth with itself, but the code it output was reasonable and it didn't make pages of markdown files either.

> I make it build things in small changes, I give very specific instructions about the architecture, and I make sure to point out existing functionality that can be used instead of writing something from scratch.

That's a bummer. I found that once the models start having problems that it cascades.

I've also been able to keep steady progress on a 70k+ LOC GUI side project without the endless whack-a-mole of bugs using Codex and Sol. Squash a bug, review architecture, move on, etc.

[whateveracct]

> So maybe there's 2x speedup in coding. But that's only a small speedup in the totality of everything software engineers do.

Amdahl's Law should be familiar to anyone with a 4y computer science/engineering degree. Why aren't they applying it to their own throughput?

[t-writescode]

Well, there’s probably several reasons:

  1) because they’re not doing those others spheres of work
  2) because they don’t think about the work
     nor how tired they are afterwards
  3) they’re proselytizing AI work as the future and
     that conflicts with that vision
  4) the other shoe hasn’t dropped
  5) they really don’t see it
  6) some people genuinely hate programming and this
     helps them skip that.

I’m sure others as well.

[ChadNauseam]

Let's apply it:

- 25% planning & aligning with other teams

- 25% coding

- 25% testing/verifying

- 25% code review/rework

I'd say that thanks to LLM assistance I'm 10x faster at coding, code review/rework, and testing/verifying. (LLMs can partially automate testing/verifying too, and the code is higher quality now as well so less testing/verifying is necessary).

So that leaves us with:

- 92.5% planning & aligning with other teams

- 2.5% coding

- 2.5% testing/verifying

- 2.5% code review/rework

Obviously, that makes zero sense as a split. If you saw any organization doing that, you'd suggest having fewer teams, more silos, etc. Maybe you have designers produce code, instead of showing the designs to coders and having the coders implement it. Maybe you force all your engineers to dogfood the product that way they can identify issues themselves rather than needing QA teams to do it. And so on. so the last category, "planning & aligning with other teams", falls too.

[steveBK123]

The other hard to measure part is that I see/hear a lot of LLM usage going towards dev work that was never prioritized before.

Suddenly devs who were cranking out features with no interest in infrastructure are attacking giant refactors to make the code more understandable to the LLM.

Other devs are using LLMs to build themselves quality of life SDLC tools completely separate from the core code base.

Plenty of other examples of this.

[leoedin]

The quality of my bash scripts is 100x what they used to be. Proper help messages, flag parsing, functions, internal variables, using sed all the time (I could never wrap my head around that).

Of course, the main issue is that they’re completely undebbugable now. My bash scripts used to be a sequential list of commands, now they’re 500 lines of variable laden functions.

Is my life any better? Dunno. But it’s satisfying (until there’s a bug)

[nasretdinov]

> the main issue is that they’re completely undebbugable now

Well, that means the quality actually dropped then :). Looking impressive isn't equal to quality, understandability and reliability is

[xboxnolifes]

100x UX quality, not DX quality.

[gmassman]

Not necessarily…. As with all engineering choices there are tradeoffs involved. There can be both advantages and disadvantages to using AI to build scripts. And more likely than not you’ll need AI to help debug the scripts when they have issues. Real tradeoffs to consider.

[mckee_plus_plus]

I switched to js for dev related scripting after seeing how much python scripting is used for portable c++ projects like chromium.

A node glob() or a regexp string.replace call is probably easier to read than spaghetti shell.

And your llm might do a much better job of creating clean, readable and testable code.

[metalspot]

llms are 100X for me, but only 10X goes into actual shipping code. the rest is all on doing the dev process the way it actually should be done: incremental POCs, refining specifications, modularization, exhaustive test suites with 95%+ logical unit test coverage, fuzzing unit test coverage, exhaustive e2e testing, with a fuzzer harness over e2e tests for long term simulation and scale testing, bechmark gated self improvement loops for performance optimization, etc. i guarantee you i can ship higher quality 100% llm generated code than any human.

[t-writescode]

It scares me how much of this is true - people truly weren't doing all of this before; but like ... some of us (like myself) were, and we were trying to push other people to write good code like that - code that helped everyone else's future velocity be faster - including our own on the next MR and the next one after that.

I'm glad you're doing all those right things; but, really, some of us were already doing most of, if not all of that. And some of us (like myself) were doing it very quickly, too, since we'd been doing it that way over a decade and constantly getting faster.

[move-on-by]

Just one more data point but this is what I’m seeing too. The extreme ‘ruthless’ prioritization has stopped and devs (who want it) have breathing room to make the enhancements they’ve been wanting all along. Turns out what is good for AI is also good for regular old school SDLC. All you have to do is say it will make AI more effective.

[senderista]

You missed documentation. There was never time for that until now.

[sensanaty]

IME LLM docs are worse than useless the majority of the time. It's like most LLM writing, an inscrutable wall of text that simultaneously says nothing and too much at the same time.

[senderista]

I agree, but there's been a push at my company for humans to write docs to help the LLMs.

[coffeebeqn]

I use CC to do the testing and verifying in multiple deployment environments. It’s faster at things like coming up a plausible test payload or ten, querying a bunch of metrics from datadog, querying and verifying state from a db before and after testing, scanning all the deployed logs for relevant errors. For me that has been more of an uplift than the coding. I still feel like communication it’s worst at. So writing tickets, getting a decision out of a group, documenting only the important things, etc. it’s still not that good at

[andrekandre]

  > - 25% planning & aligning with other teams
  > - 25% coding
  > - 25% testing/verifying
  > - 25% code review/rework

legit question: has anyone got ai to do all the above? i've heard people trying ai-dlc [0] but it sounds like a slog...

[0] https://aws.amazon.com/jp/blogs/devops/ai-driven-development...

[agumonkey]

Agentic tools changed the workflow in our org, we were very tense before with a simple yet careful team process.. Since last winter we're now pushing a lot more but the teamwork (which was brittle before) is now mostly gone, everybody can roll on its own, but the review process didn't scale and now monthly deliveries are full of "seems to work". nobody checks properly, nobody reviews, nobody tighten any bolts.. and nobody cares much anyway, the system allows it.