|
For the past 18 months, I've been creating an in-house GUI application which is starting to approaching Photoshop-level complexity. By which I mean: it's still probably a solid order of magnitude less complex than Photoshop -- but it's not two orders of magnitude less complex. It's several orders of magnitude more complex than the examples of vibecoded apps I typically see. (The domain, FWIW, is a geospatial transport-planning tool, including a completely custom microsimulation engine, with loads of options for visualization, analytics, etc.) At the start of this development process, LLMs were capable of assisting with little more than the framework boilerplate stuff. That was very useful, but was well under 50% of the LOC. They were particularly bad at understanding the microsimulator, where they would routinely forget which end of a FIFO queue was the front. LLMs are routinely and correctly criticized for their lack of a true world model, and when it came to modelling real-world physical/spatial/geographic systems, the fact that they see the world as nothing but text was a huge limitation. Not just in terms of having a pretty hazy grasp on concepts like "spatial direction", but even more critically, being unable to rationalize about the "world-within-a-world" which the simulator is attempting to model. They were fully unable to do that. That was 18 months ago. Now, Claude writes > 99% of my code. It demonstrates a far better grasp of first-order world-model phenomena (like "spatial orientation"), and a decent (but not fantastic) ability to reason about the second-order "world-within-a-world" that the simulator is creating. It's a huge improvement. For some areas of the code, I still need to spell things out very explicitly, giving precise instructions for how a method will work. That's definitely not vibe-coding. But for other areas of the code, I can just say "add this analysis or visualization feature", without specifying how, and Claude will one-shot a result that's somewhere between good and great. So where we're at now is that Claude often needs hand-holding for some of the most complex areas of the code, and it definitely doesn't understand how the whole application hangs together -- I have to keep reminding it of that, and am constantly taking steps to ensure that it remains well-architected doesn't devolve into a collection of warring patches. And yet -- in the past 18 months, the boundary between what the LLM is capable of and what I need to exercise control over has shifted MASSIVELY, and it has shifted in the direction of LLMs being more able to rationalize about meta-models and higher-order architectures. I've got two small children. When they say they can't do something, I always remind them that they can't do that thing -- YET. What they can do today is very far from the ultimate limits of their capabilities. I feel similarly about the capabilities of LLMs. No, they definitely can't vibecode a Photoshop-class application. YET. |