[dfabulich]

We've seen this happen over and over again, when a new leaky layer of abstraction is developed that makes it easier to develop working code without understanding the lower layer.

It's almost always a leaky abstraction, because sometimes you do need to know how the lower layer really works.

Every time this happens, developers who have invested a lot of time and emotional energy in understanding the lower level claim that those who rely on the abstraction are dumber (less curious, less effective, and they write "worse code") than those who have mastered the lower level.

Wouldn't we all be smarter if we stopped relying on third-party libraries and wrote the code ourselves?

Wouldn't we all be smarter if we managed memory manually?

Wouldn't we all be smarter if we wrote all of our code in assembly, and stopped relying on compilers?

Wouldn't we all be smarter if we were wiring our own transistors?

It is educational to learn about lower layers. Often it's required to squeeze out optimal performance. But you don't have to understand lower layers to provide value to your customers, and developers who now find themselves overinvested in low-level knowledge don't want to believe that.

(My favorite use of coding LLMs is to ask them to help me understand code I don't yet understand. Even when it gets the answer wrong, it's often right enough to give me the hints I need to figure it out myself.)

[danielmarkbruce]

LLMs don't create an abstraction. They generate code. If you are thinking about LLMs as a layer of abstraction, you are going to have all kinds of problems.

[sanswork]

My C compiler has been generating assembly code for me for 30 years. And people were saying the same thing even earlier about how compilers and HLLs made developers dumb because they couldn't code in asm.

[__loam]

C compilers are deterministic. You don't have to inspect the assembly they produce to know that it did the right thing.

[sanswork]

And C compilers have bugs too earlier ones a lot more though thankfully we are now several decades into their development.

[elicksaur]

Very whataboutism. Do you understanding why the person is making this argument? Or is everything just postmodern deconstructionism?

Everything has some imperfection, so LLMs are just fine... Completely missing the totally valid criticisms people have of the systems.

[Veserv]

I like to call that “unary thinking”. Nothing is “perfect”, therefore everything is “imperfect”, so everything is the “same”. One category for everything, unary.

[sanswork]

You're trying too hard to argue here.

Every 5 years or so of my career something new has come out to make coding easier and every time a stack of people like yourself come out to argue it's making devs dumb or worse.

LLMs still generate terrible output a lot of the time but they are improving. Early compilers generated terrible ASM a lot of the time to the point that it was common to use inline assembly in your code or rewrite parts later. Tools can improve, the point is that neither make the dev worse they just add to productivity.

Writing code isn't my job it's a task I do to make the systems I design functional.

[lurking_swe]

critical thinking is hard i guess

[danielmarkbruce]

Presumably you don't throw out the c code and just check in the assembly.

[elicksaur]

>And people were saying

Source? A quote? Or are we just making up historical strawmen to win arguments against?

[sshine]

I've heard from people older than me that this is how people felt about compilers.

https://vivekhaldar.com/articles/when-compilers-were-the--ai...

He provides some sources (7) at the bottom of the article.

To pick one, the following has a video interview with one of the founders of Fortran:

https://www.ibm.com/history/fortran#:~:text=Fortran%20was%20...

So we're not making up stuff, this perspective was ubiquitous among assembly programmers of the 1950s. In 1958 (as the first article I link to mentions), half of programs were written in Fortran. Which means half of people still thought writing assembly by hand was the way to go.

I've personally written assembly by hand for money on an obscure architecture, and I've also written a non-optimizing compiler for a subset of Rust to avoid the assembly. There is great joy in playing stack tetris, but changing code requires a lot of effort. Imagine if there weren't great alternatives, you'd just get good at it.

[YeGoblynQueenne]

I imagine if there weren't compilers (or interpreter) I would never have learned how to code. My generation of programmers was taught with Java and in my university course we did all our homework in the first year using BlueJay, a program that made it _even easier_ to get up and running with a bit of Java code.

(just to save some face: I learned Prolog in my second year).

[sshine]

Yeah, before high-level languages, programming was mostly done by mathematicians and electrical engineers who felt adventurous.

I also had BlueJay on my first semester. But fortunately I had machine architecture, compilers and operating systems later.

[YeGoblynQueenne]

Not quite what the OP claims but see for example the Story of Mel:

  I had been hired to write a FORTRAN compiler
  for this new marvel and Mel was my guide to its wonders.
  Mel didn't approve of compilers.

  ``If a program can't rewrite its own code'',
  he asked, ``what good is it?''

https://www.gutenberg.org/cache/epub/3008/pg3008-images.html...

Though to be fair The Story of Mel supports rather than refute the argument against compilers.

[sanswork]

https://en.m.wikipedia.org/wiki/Real_Programmers_Don%27t_Use...

The joke story is mocking the common arguments/beliefs at the time.

If you expect me to source you a collection of comments about "real programmers" from over 30 years ago though that is too much of an ask but I was there, I read it often and I started fairly late on the scene in the 90s.

[HeyLaughingBoy]

No quote or source, but I can corroborate. I've even heard the same argument when C++ was getting popular and C was still the "standard."

[lurking_swe]

except we can guarantee (with tests) that the generated instructions from the compiler are bug free 99% of the time. Pretty big difference there.

[sanswork]

The tool currently being bad doesn't matter to the argument of whether or not they make devs dumb.

[lurking_swe]

if i can one day have a similar level of confidence in LLM output, as i do in a compiler, then i will also call it an abstraction. until then…i wait. :)

But i agree, it’s unrelated to devs being dumb.

[Sparkyte]

They can also generate documentation of code you've written. So it is very useful if leveraged correctly to understand what the code is doing. Eventually you learn all of the behaviors of that code and able to write it yourself or improve on it.

I would consider it as a tool to teach and learn code if used appropriately. However LLMs are bullshit if you ask it to write something, pieces yes, whole code... yeah good luck having it maintain consistency and comprehension of what the end goal is. The reason it works great for reading existing code is that the input results into a context it can refer back to but because LLMs are weighted values it has no way to visualize the final output without significant input.

[haydenlingle]

This is a disingenuous critique of what was said.

The point is LLMs may allow developers to write code for problems they may not fully understand at the current level or under the hood.

In a similar way using a high level web framework may allow a developer to work on a problem they don’t fully understand at the current level or under the hood.

There will always be new tools to “make developers faster” usually at a trade off of the developer understanding less of what specifically they’re instructing the computer to do.

Sometimes it’s valuable to dig and better understand, but sometimes not. And always responding to new developer tooling (whether LLMs or Web Frameworks or anything else) by saying they make developers dumber, can be naive.

[danielmarkbruce]

Nope, it's not disingenuous. It's a genuine critique that it's just stupid way to think about things. You don't check in a bunch of prompts, make changes to the prompts, run them through a model, compile/build the code.

It's simply not the same thing as a high level web framework.

If you have an intern, or a junior engineer - you give them work and check the work. You can give them work that you aren't an expert in, where you don't know all the required pieces in detail, and you won't get out of it the same as doing the work yourself. An intern is not a layer of abstraction. Not all divisions of labor are via layers of abstraction. If you treat them all that way it's dumb and you'll have problems.

[simonw]

Leaky abstractions is a really appropriate term for LLM-assisted coding.

The original "law of leaky abstractions" talked about how the challenge with abstractions is that when they break you now have to develop a mental model of what they were hiding from you in order to fix the problem.

(Absolutely classic Joel Spolsky essay from 22 years ago which still feels relevant today: https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a... )

Having LLMs write code for you has a similar effect: the moment you run into problems, you're going to have to make sure you deeply understand exactly what they have done.

[gopher_space]

> Having LLMs write code for you has a similar effect: the moment you run into problems, you're going to have to make sure you deeply understand exactly what they have done.

I'm finding that, If I don't have solid mastery of at least one aspect of generated code, I won't know that I have problems until they touch a domain I understand.

[keybored]

[deleted]

[simonw]

Leaky abstractions doesn't imply abstractions are bad.

Using abstractions to trade power for simplicity is a perfectly fine trade-off... but you have to bare in mind that at some point you'll run into a problem that requires you to break through that abstraction.

I read that essay 22 years ago and ever since then I've always looked out for opportunities to learn little extra details about the abstractions I'm using. It pays off all the time.

[keybored]

> Leaky abstractions doesn't imply abstractions are bad.

No, of course not. I’m saying that the article is bad. Because he’s not saying anything that the term itself isn’t already describing or implying.

No adjectives needed. And certainly no laws.

[bakugo]

Those aren't the same thing at all, and you already mentioned why in your comment: leakiness. The higher up you go on the abstraction chain, the leakier the abstractions become, and the less viable it is to produce quality software without understanding the layer(s) below.

You can generally trust that transistors won't randomly malfunction and give you wrong results. You can generally trust that your compiler won't generate the wrong assembly, or that your interpreter won't interpret your code incorrectly. You can generally trust that your language's automatic memory management won't corrupt memory. It might be useful to understand how those layers work anyway, but it's usually not a hard requirement.

But once you reach a certain level of abstraction (usually 1 level above programming language), you'll start running into more and more issues resulting from abstraction leaks that require understanding the layer below to properly fix. Probably the most blatant example of this nowadays are "React developers" who don't know JS/CSS/HTML and WILL constantly be running into issues that they can't properly solve as a result, and are forced to either give up or write the most deranged workarounds imaginable that consist of hundreds of lines of unintelligible spaghetti.

AI is the highest level of abstraction so far, and as a result, it's also the leakiest abstraction so far. You CANNOT write proper functional and maintainable code using an LLM without having at least a decent understanding of what it's outputting, unless you're writing baby's first todo app or something.

[ToucanLoucan]

> But once you reach a certain level of abstraction (usually 1 level above programming language), you'll start running into more and more issues resulting from abstraction leaks that require understanding the layer below to properly fix. Probably the most blatant example of this nowadays are "React developers" who don't know JS/CSS/HTML and WILL constantly be running into issues that they can't properly solve as a result, and are forced to either give up or write the most deranged workarounds imaginable that consist of hundreds of lines of unintelligible spaghetti.

I want to frame this. I am sick to death of every other website and application needing a gig of RAM and making my damn phone hot in my hand.

[skoodge]

Not all of those abstractions are equally leaky though. Automatic memory management for example is leaky only for a very narrow set of problems, in many situations the abstraction works extremely well. It remains to be seen whether AI can be made to leak so rarely (which does not meant that it's not useful even in its current leaky state).

[notTooFarGone]

If we just talk in analogies: a cup is also leaky because fluid is escaping via vapours. It's not the same as a cup with a hole in it.

Llms currently have tiny holes and we don't know if we can fix them. Established abstractions are more like cups that may leak but only in certain conditions (when it's hot)

[gtsop]

> But you don't have to understand lower layers to provide value to your customers, and developers who now find themselves overinvested in low-level knowledge don't want to believe that.

This is the weakest point which breaks your whole argument.

I see it happening ALL the time: newer web developers enter the field from an angle of high abstraction, whenever these abstractions don't work well, then they are completelly unable to proceed. They wouldn't be in that place if they knew the low-level and it DOES prevent them from delivering "value" to their customers.

What is even worse than that, since these developers don't understand exactly why some problen manifests, and the don't even understand exactly what their abstraction trully solves, they wrongly proceed to solve a problem using the wrong (high level) tools.

[huxley]

Yeah … but

That has some amount to do with the level abstraction but almost everything to do with inexperience. The lower level you get, the harder the impact of inexperience.

New web developers are still sorting themselves out and they are at a stage where they’ll suck no matter what the level of abstraction.

[gtsop]

I get what you say but I must insist. Abstractions don't properly surface the underlying causes when they break.

This creates a barier in gaining experience on the topic that matters in order to solve your problem. Let me try to give an example (please don't neat pick, it is just an example):

A front-end dev who has never bundled an app like we did in the old days (manual scripts or manual gulp, grunt, webpack pipelines) but use a off-the-shelf webpack config (lets say CRA) has great trouble understanding why some webpack plugin doesn't work as expected, or they might not even understand how importing an svg into their jsx actually works. Yes this is due to lack of experience, but their current level of working isn't exposing those details directly so they can't gain experience, the waters are way too deep.

[dragonwriter]

> Abstractions don't properly surface the underlying causes when they break.

"Abstractions" in the abstract neither do nor do not do this, as it is orthogonal to what an abstraction is; concrete implementations of abstractions can either swallow or wrap information about the underlying cause when they break, both are valid approaches.

> but their current level of working isn't exposing those details directly so they can't gain experience

This isn't really true. They absolutely can gain experience; they don't pay a (very high) extra effort with the side benefit of maybe gaining some experience on routine tasks where nothing breaks, and tend to gain experience only if they expend extra effort in situations where things fail or their are unusual challenges to resolve.

[gtsop]

> They absolutely can gain experience

Not sure where you are getting with this. Yes, literally speaking they can, but what matters is if they actually do, in general, in the real world. My admitedly biassed opinion, based on personaly observating other colleagues, shows that in general, they don't learn and they give up.

Have you noticed an other trend in your circles?

[dragonwriter]

> Not sure where you are getting with this. Yes, literally speaking they can, but what matters is if they actually do,

And, despite the fairly consistent bias people used to working at lower levels of abstraction have against people who usually work at higher levels, we all work with some level of abstraction, and most people gain experience at lower levels by dealing with problems that emerge, and those that don't don't fail to do so because they can't but because they choose not to bother either because they have other people that they can rely on for lower-level problems, or because they aren't interested and are doing things where they can afford to simply change what they are doing in response to those problems.

[keybored]

And every time the commentariat dismisses it with the trope that it’s the same as the other times.

It’s not the same as the other times. The naysayers might be the same elitists as the last time. But that’s irrelevant because the moment is different.

It’s not even an abstraction. An abstraction of what? It’s English/Farsi/etc. text input which gets translated into something that no one can vouch for. What does that abstract?

You say that they can learn about the lower layers. But what’s the skill transfer from the prompt engineering to the programming?

People who program in memory-managed languages are programming. There’s no paradigm shift when they start doing manual memory management. It’s more things to manage. That’s it.

People who write spreadsheet logic are programming.

But what are prompt engineers doing? ... I guess they are hoping for the best. Optimism is what they have in common with programming.

[101008]

> (My favorite use of coding LLMs is to ask them to help me understand code I don't yet understand. Even when it gets the answer wrong, it's often right enough to give me the hints I need to figure it out myself.)

Agree, specially useful when you join a new company and you have to navegate a large codebase (or bad-maintained codebase, which is even worse by several orders of magnitude). I had no luck asking LLM to fix this or that, but it did mostly OK when I asked how it works and what the code is trying to code (it includes mistakes but that's fine, I can see them, which is different if it was just code that I copy and paste).

[elicksaur]

See this as one of the major selling points.

…but I haven’t joined a new company since LLMs were a thing. How often is this use case necessary to justify $Ts in investment?

[101008]

No ieda about that, those are amounts of money I can't even consider because I couldn't tell the different of T vs B or even a hundred millions (as someone who never had more than 100k).

It's something I would cosnider paying for the first months when joining a new company (specially with a good salary), but not more than that, to be honest.

[happytoexplain]

This "it's the same as the past changes" analogy is lazy - everywhere it's reached for, not just AI. It's basically just "something something luddites".

Criticisms of each change are not somehow invalid just because the change is inevitable, like all the changes before it.

[notarobot123]

When a higher level of abstraction allows programmers to focus on the detail relevant to them they stop needing to know the low level stuff. Some programmers tend not to be a fan of these kinds of changes as we well know.

But do LLMs provide a higher level of abstraction? Is this really one of those transition points in computing history?

If they do, it's a different kind to compilers, third-party APIs or any other form of higher level abstraction we've seen so far. It allows programmers to focus on a different level of detail to some extent but they still need to be able to assemble the "right enough" pieces into a meaningful whole.

Personally, I don't see this as a higher level of abstraction. I can't offload the cognitive load of understanding, just the work of constructing and typing out the solution. I can't fully trust the output and I can't really assemble the input without some knowledge of what I'm putting together.

LLMs might speed up development and lower the bar for developing complex applications but I don't think they raise the problem-solving task to one focused solely on the problem domain. That would be the point where you no longer need to know about the lower layers.

[vishalontheline]

Last year I learned a new language and framework for the first time in a while. Until I became used to the new way of thinking, the discomfort I felt at each hurdle was both mental and physical! I imagine this is what many senior engineers feel when they first begin using an AI programming assistant, or an even more hands-off AI tool.

Oddly enough, using an AI assistant, despite it guessing incorrectly as often as it did, helped me learn and write code faster!

[player1234]

How much faster? How did you measure it?

[jaredklewis]

I think it’s usually helpful if your knowledge extends a little deeper than the level you usually work at. You need to know a lot about the layer you work in, a good amount about the surrounding layers, and maybe just a little bit about more distant layers.

If you are writing SQL, it’s helpful to understand how database engines manage storage and optimize queries. If you write database engine code, it’s helpful to understand (among many other things of course) how memory is managed by the operating system. If you write OS code, it’s helpful to understand how memory hardware works. And so on. But you can write great SQL without knowing much of anything about memory hardware.

The reverse is also true in that it’s good to know what is going on one level above you as well.

Anyway my experience has been that knowledge of the adjacent stack layers is highly beneficial and I don’t think exaggerated.

[djaouen]

When a lower layer fails, your ability to remedy a situation depends on your ability to understand that layer of abstraction. Now: if an LLM produces wrong code, how do you know why it did that?

[GoblinSlayer]

Understanding assembler (or even just ABI) if useful.

[Sparkyte]

Let me alter this perspective, you can use it to learn why parts of code does and use it for commenting. Help you read what might be otherwise unreadable. LLMs and programming is good, but not great. However it can easily be someone to teach a developer what parts of the code they are working on.

[yubblegum]

The difference between a mechanic and an engineer, illustrated ..

[yapyap]

Sounds like you’re coping cause you have some type of investment in LLM “coding”, whether that is financial or emotional.

I won’t waste my time too much reacting to this nonsensical comment but I’ll just give this example, LLMs can hallucinate, where they generate code that’s not real, LLMs don’t work off straight rules, they’re influenced by a seed. Normal abstraction layers aren’t.

I dearly hope you’re arguing in bad faith, otherwise you are really deluded with either programming terms or reality.

[intelVISA]

Abstraction is fine, it allows you to work faster, or easier. Reliance that becomes dependency is the problem - when abstraction supersedes fundamentals you're no longer able to reason about the leaks and they become blindspots.

Don't confuse low-level tedium with CS basics, if you're arguing that knowing how computers work is not relevant to working as a SWE then sure, but why would a company want a software dev that doesn't seem to know software? Usually your irreplaceable value as a developer is knowing and mitigating the leaks so they don't wind up threatening the business.

This is where the industry most suffers from not having a standardized-ish hierarchy, you're right that most shops don't need a trauma surgeon on-call for treating headaches but there's still many medical options before resorting to random grifter who simply "watched some Grey's Anatomy" as "medschool was a barrier for providing value to customers".