[wiremine]

> I suspect that there is a strong correlation between programmers who don't think that there needs to be a model/theory, and those who are reporting that LLMs are speeding them up.

I also strongly agree with Lamport, but I'm curious why you don't think Ai can help in the "theory building" process, both for the original team, and a team taking over a project? I.e., understanding a code base, the algorithms, etc.? I agree this doesn't replace all the knowledge, but it can bridge a gap.

[wholinator2]

I agree, the llm _vastly_ speeds up the process of "rebuilding the theory" of dead code, even faster than the person who wrote it 3 years ago can. I've had to work on old fortran codebases before and recently had the pleasure of including ai in my method and my god, it's so much easier! I can just copy and paste every relevant function into a single prompt, say "explain this to me" and it will not only comment each line with its details, but also elucidate the deeper meaning behind the set of actions. It can tell exactly which kind of theoretical simulation the code is performing without any kind of prompting on my part, even when the functions are named things like "a" or "sv3d2". Then, i can request derivations and explanations of all relevant theory to connect to the code and come away in about 1 days worth of work with a pretty good idea of the complete execution of a couple thousand lines of detailed mathematical simulations in a languages I'm no expert in. LLMs contribution to building theory has actually been more useful to me than is contribution in writing code!

[bunderbunder]

From what I've seen they're great at identifying trees and bad at mapping the forest.

In other words, they can help you identify what fairly isolated pieces of code are doing. That's helpful, but it's also the single easiest part of understanding legacy code. The real challenges are things like identifying and mapping out any instances of temporal coupling, understanding implicit business rules, and inferring undocumented contracts and invariants. And LLM coding assistants are still pretty shit at those tasks.

[manishsharan]

Not always.

You could paste your entire repo into Gemini and it could map your forest and also identify the "trees".

Assuming your codebase is smaller than Gemini context window. Sometimes it makes sense to upload a package,s code into Gemini and have it summarize and identify key ideas and function. Then repeat this for every package in the repository.then combine the results . It sounds tedious but it is a rather small python program that does this for me.

[bunderbunder]

I've tried doing things like that. Results reminded me of that old chestnut, "Answers: $1. Correct answers: $50."

Concrete example, last week a colleague of mine used a tool like this to help with a code & architectural review of a feature whose implementation spanned four repositories with components written in four different programming languages. As I was working my way through the review, I found multiple instances where the information provided by the LLM missed important details, and that really undermined the value of the code review. I went ahead and did it the old fashioned way, and yes it took me a few hours but also I found four defects and failure modes we previously didn't know about.

[prmph]

Indeed, I once worked with a developer on a contract team who was only concerned with runtime execution, no concern whatever for architecture or code clarity or whatever at all.

The client loved him, for obvious reasons, but it's hard to wrap my head around such an approach to software construction.

Another time, I almost took on a gig, but when I took one look at the code I was supposed to take over, I bailed. Probably a decade would still not be sufficient for untangling and cleaning up the code.

True vibe coding is the worst thing. It may be suitable for one-ff shell script s of < 100 line utilities and such, anything more than that and you are simple asking for trouble

[N70Phone]

> I.e., understanding a code base, the algorithms, etc.?

The big problem is that LLMs do not *understand* the code you tell them to "explain". They just take probabilistic guesses about both function and design.

Even if "that's how humans do it too", this is only the first part of building an understanding of the code. You still need to verify the guess.

There's a few limitations using LLMs for such first-guessing: In humans, the built up understanding feeds back into the guessing, as you understand the codebase more, you can intuit function and design better. You start to know patterns and conventions. The LLM will always guess from zero understanding, relying only on the averaged out training data.

A following effect is that which bunderbunder points out in their reply: while LLMs are good at identifying algorithms, mere pattern recognition, they are exceptionally bad at world-modelling the surrounding environment the program was written in and the high level goals it was meant to accomplish. Especially for any information obtained outside the code. A human can run a git-blame and ask what team the original author was on, an LLM cannot and will not.

This makes them less useful for the task. Especially in any case where you intent to write new code; Sure, it's great that the LLM can give basic explanations about a programming language or framework you don't know, but if you're going to be writing code in it, you'd be better off taking the opportunity to learn it.

[netghost]

Perhaps it's the difference between watching a video of someone cooking a meal and cooking it for yourself.

[wiremine]

That's a good analogy.

To clarify my question: Based on my experience (I'm a VP for a software department), LLMs can be useful to help a team build a theory. It isn't, in and of itself, enough to build that theory: that requires hands-on practice. But it seems to greatly accelerate the process.

[panarky]

People always wring their hands that operating at a new, higher level of abstraction with destroy people's ability to think and reason.

But people still think and reason just fine, but now at a higher level that gives them greater power and leverage.

Do you feel like you're missing something when you "cook for yourself" but you didn't you didn't plant and harvest the vegetables, raise and butcher the protein, forge the oven, or generate the gas or electricity that heats it?

You also didn’t write the CPU microcode or the compiler that turns your code into machine language.

When you cook or code, you're already operating on top of a very tall stack of abstractions.

[JoeAltmaier]

Nah. This is a different beast entirely. This is removing the programmer from the arena, so they'll stop having intuition about how anything works or what it means. Not more abstract; completely divorced from software and what it's capable of.

Sure, manager-types will generally be pleased when they ask AI for some vanilla app. But when it doesn't work, who will show up to make it right? When they need something more complex, will they even know how to ask for it?

It's the savages praying to Vol, the stone idol that decides everything for them, and they've forgotten their ancestors built it and it's just a machine.

[mikestorrent]

I agree with your sentiment. The thing is, in the past, the abstractions supporting us were designed for our (human) use, and we had to understand their surface interface in order to be able to use them effectively.

Now, we're driving such things with AI; it follows that we will see better results if we do some of the work climbing down into the supporting abstractions to make their interface more suitable for AI use. To extend your cooking metaphor, it's time to figure out the manufactured food megafactory now; yes, we're still "cooking" in there, but you might not recognize the spatulas.

Things like language servers (LSPs) are a step in this direction: making it possible to interact with the language's parser/linter/etc before compile/runtime. I think we'll eventually see that some programming languages end up being more apropos to efficiently get working, logically organized code out of an AI; whether that is languages with "only one way to do things" and extremely robust and strict typing, or something more like a Lisp with infinite flexibility where you can make your own DSLs etc remains to be seen.

Frameworks will also evolve to be AI-friendly with more tooling akin to an LSP that allows an MCP-style interaction from the agent with the codebase to reason about it. And, ultimately, whatever is used the most and has the most examples for training will probably win...

[epgui]

The comment you're replying to is not about abstraction at all. ie.: The difference between passive listening and active learning is not abstraction.

[jquaint]

I agree with this sentiment. Perhaps this is why there is such a senior / junior divide with LLM use. Seniors already build their theories. Juniors don't have that skill.