[jadar]

> So, why can't models do software maintainability?

I feel like the explanation does nothing to actually elucidate why models can't do it. Is it an inherent weakness of LLMs? Training processes? The typical "this is crap" that we constantly hear? It goes on to write about RL and how there's no penalty for bad design. But that sort of side-steps the question and makes you ask: "why not do RL and make a penalty for bad design?" Of course the models aren't good at it ... they're not good at anything until you've tuned them and put them in a harness that rewards good edits and throws away (improves) bad edits. That doesn't explain why "models can't do software maintainability." The real question is why harnesses can't do software maintainability, and how to build a system that can do it. (I suppose that's the purpose of the ad at the bottom of the page.)

[rom1504]

I think the simple answer is LLM cannot do long term planning.

Maintainability means thinking what will happen to this code in the next year and possibly hundreds of changes and based on that selecting the right abstractions that will work in the long term.

LLMs currently cannot do long term planning and specifically cannot pick abstractions that will work in the long term.

I think that's the important problem to solve. Teaching LLMs how to pick the right abstractions.

[satvikpendem]

Why can they not do long term planning? They know what good software design looks like and can code accordingly. Using Fable for a greenfield project for example, it doesn't dump everything into one file but instead does indeed split out the work into logical units as needed.

[rom1504]

Why is a very good question indeed! My guess is we have not figured out how to include long term enough tasks into the RL training.

As to whether they can today, it's pretty easy to figure out that they can't by observing the difference between asking for very high level things like "please make me a website that will be very successful" vs sending it a detailed plans on the goals and many of the specifics and then holding it accountable towards the planning.

Doing appropriate project structure and trying to optimize code quality are definitely good methods to try and increase the chance that the project will succeed long term but I would say they are only methods towards the long term goal and the LLM need to be thinking of the long term goal and how to plan towards that to achieve a much wider and subtile range of methods to achieve long term goals.

[satvikpendem]

/goal and /workflow exists. As I said in another comment, models can do it, as in they have the capacity to do so. It's just the harness that needs to be wired up correctly.

[Fordec]

I have a hunch it comes down to the training data. We have spent decades as an industry talking up the new shiny thing, deliverables, frameworks, features, performance improvements, the one thing you don't read copious amounts of prose about is "how I maintained our system so that nobody noticed".

[Terr_]

"Here's me throwing code away because it's inherently bad or no longer suitable for the new purpose."

[jdlshore]

I think it’s because maintainability is a factor of software design, and good design is squishy and hard to articulate. There’s no good metrics for design. All the tools that purport to measure it are doing trivial things like cyclomatic complexity, and can only tell you when things have gotten really really bad, but not distinguish between “good” and “pretty bad.”

Design skills take about 10 years to develop, in my experience, and a lot of smart engineers go through a phase where they make gawdawful messes in the name of “magically easy” designs around the 5 year mark.

Design requires wisdom born of experience, isn’t measurable, and requires forming a mental model of how the system hangs together as a whole. It’s just not stuff LLMs are good at.

[jadar]

> It’s just not stuff LLMs are good at.

But then you're back at the question from the OP of "why not?" Is it lack of good examples in the training data? Is there something of code "goodness" that goes beyond phenomenon that are measurable? Is it that LLMs lack in the ability to organize and think abstractly?

[jdlshore]

All of the above, but mostly the latter, IMO.

[zahrevsky]

> why not do RL and make a penalty for bad design?

But this is clearly explained in the article. We can tell whether the design was good or bad only weeks or even months later, which is too long to wait for artificial trajectories. Tests, by contrast, provide validation in seconds or minutes.

[dhorthy]

fair point, this is the thing I struggled most to extract out while writing it - if you can propose an RL environment that penalizes a model for bad design, then I'm all ears - right now there's no fast oracle/verifier for this (as stated in the post)

My current evolving take on "how would you build such a thing" is you need to tee up a roadmap of 20 features and feed them to a model one at a time, so it can't design up front for what's coming.

That way if it builds the first 10 features and the codebase goes to slop, it get's penalized when it can't build features 11-20, or when those features take wayyy more tokens/time/cycles than a model that maintains a clean codebase can do.

This is how most real software is built by most teams - incrementally, getting feedback from users along the way, and steering goals in response.

[jadar]

Isn't that how AI written software gets better too? By steering the model towards the goals of the user?

I don't know if it's a question of how many features to feed to the model, either. Of course, overwhelm the context with too many features and it will get confused. But that's where the memory management idea that G. Huntley talks about is helpful. You're trying to steer the model within its memory limits towards a certain goal.

The problem is getting it to produce "good" code. Formalizing what that means is the task of the programmer. How do you steer a model towards always, or more often, producing good code so that you don't have to do rework? That's the same problem as with a junior engineer, but the way you do it is different. Right now we're trying to do it with mountains of prompts — which sort of works but has diminishing returns — and with onerous code reviews. We've seen this get better over time, but I think some more mechanical methods will help as we figure out how best to steer the models.

[goyozi]

I’ve experimented a bit with this approach for quasi-benchmarking models. Most models nowadays will keep trying until you cut them off, they’re happy to go into millions of tokens for an easy task, digging deeper and deeper into crap design. Compared to a clean reference implementation you get easily 20x token difference at task 11+ for the same model. Somewhat ironically I never published the results because the both the harness and the task set were vibe coded and I didn’t have the energy to clean it all up.