[appease7727]

The way it works for me at least is I can fit a huge amount of context in my head. This works because the text is utterly irrelevant and gets discarded immediately.

Instead, my brain parses code into something like an AST which then is represented as a spatial graph. I model the program as a logical structure instead of a textual one. When you look past the language, you can work on the program. The two are utterly disjoint.

I think LLMs fail at software because they're focused on text and can't build a mental model of the program logic. It take a huge amount of effort and brainpower to truly architect something and understand large swathes of the system. LLMs just don't have that type of abstract reasoning.

[starlust2]

It's not that they can't build a mental model, it's that they don't attempt to build one. LLMs jump straight from text to code with little to no time spent trying to architect the system.

[taminka]

i wonder why nobody bothered w/ feeding llms the ast instead (not sure in what format), but it only seems logical, since that's how compilers undestand code after all...

[NitpickLawyer]

There are various efforts on this, from many teams. There's AST dump, AST-based graphs, GraphRAG w/ AST grounding, embeddings based AST trimming, search based AST trimming, ctags, and so on. We're still in the exploration space, and "best practices" are still being discovered.

It's funny that everyone says that "LLMs" have plateaued, yet the base models have caught up with early attempts to build harnesses with the things I've mentioned above. They now match or exceed the previous generation software glue, with just "tools", even with limited ones like just "terminal".