Hacker News new | ask | show | jobs
by Incipient 31 days ago
I have spent a SOLID 3 full days 8h/day (plus long running tasks overnight) thrashing out a random idea for a Web application using purely Opus (mostly Max, sometimes ultracode version). I'm not a project manager, but I genuinely tried a full 3-tier spec out - design->specs->build details.

While it was significantly better than previous attempts, it still misses very basic things - sporadically. Eg. A clear design requirement was essentially adding clients, explained clearly and comprehensively. The ability to add clients was entirely missed in the build and iteration (there were multiple 'please check its all done' separate agent runs/checks).

I can imagine in a fully autonomous deployment, in even moderate complexity, even to this day would still occasionally mess up - badly enough to cause non-trivial business issues.

I haven't managed to really figure out what's the best way, but my latest thinking is really having boil down tasks to almost unit operations "add UI button, wire to Api call. End".

4 comments

> I haven't managed to really figure out what's the best way, but my latest thinking is really having boil down tasks to almost unit operations "add UI button, wire to Api call. End".

And at that point you might as well just code the thing yourself.

> I haven't managed to really figure out what's the best way

For you, the best way is to break your code down into modules insofar as possible, so that you don't overrun the context window. Opus Max starts forgetting things the minute it begins compressing your conversation -- and multiple compressions can make for gaps in memory.

I find that it's also important to have another model serve as review/critique. I use Opus Max for code and 5.5 Pro for immediate code review. The latter will often pick up on things that might have been missed, and will usually provide good suggestions.

I've made it write a /status-line thing to display context tokens in the status line and also a hook to stop and ask to continue or compact whenever it reaches 250k tokens. For subscription I have also made it stop at 90% usage so Claude chat is not unusable between coding sessions. The greatest addition so far.
Hard to say exactly what went wrong from outside, but a frontier model not being able to implement a simple CRUD feature after 3x8 = 24 hours of work isn't "it can't do this". Let me hazard a guess from what you wrote. The 3-tier spec (design → specs → build details) may be the cause rather than the cure. A big upfront spec has two failure modes the model can't help you with: it can quietly contain contradictions, and it can be ambiguous in ways the model resolves by guessing instead of asking. "Adding clients" is a good example of the trap, even assuming your real spec was more detailed than the comment. "Client" is overloaded — a customer in the domain? An API client? A consumer of a service? And "A clear design requirement was essentially adding clients" is very imprecise: does the model add them, or build a UI so the end user can add them? I know this was just your comment and sorry to sound harsh but if the spec had sentences like that I can definitely see it going off the rails.

Your own conclusion, smaller, concrete units, is the right direction. Except by units I don't mean partitioing the program into smaller units (files, modules). In fact, you should stop thinking about implementation at all. I'm thinking more about the way of asking the LLM to build it. One feature at a time etc. so you can tighten the feedback loop. Then you can early on (in the first hour say): "I also need a way that the user can add/manage clients - basic CRUD" and that small sentence might be enough the model makes it all (UI, API, backend etc.) to enable that and put it in a proper place in the app. A big ambiguous spec defers that discovery to the worst possible moment.

> And "A clear design requirement was essentially adding clients" is very imprecise

That's not a sentence they literally gave the agent. This is their full quote.

> A clear design requirement was essentially adding clients, explained clearly and comprehensively.

Why are you assuming they described this so ambiguously?

> there were multiple 'please check its all done' separate agent runs/checks

You could ask it to go through the spec point by point and then mark what is done and WHERE/WHY, then it'd point you towards exactly what might be missing.