[jiggawatts]

I was just thinking of this exact comic, which is etched into my brain for some reason.

The obvious counter to this is that we've been going through this evolution of increasing abstraction as developers for nearly a century now.

In the 40s and well into the 60s, most code was written either as straight up machine code or an assembly language. MS DOS is almost entirely assembly.

UNIX ushered in the era of "high level" portable languages like C, Fortran, and Pascal that some developers hated because they felt like they were losing the fine-grained control that they had with assembly. The compilers just "weren't as good" as humans at optimisation!

Then the compilers got better and people started using garbage-collected languages like Perl, Python, Java, JavaScript, and C#. Similarly, many people bemoaned the lack of control over memory allocation, lower efficiency, etc.

We're simply stepping up to the next level of abstraction.

Look at it this way: decades ago when I first discovered C++ templates, it felt like waving a magic wand in the direction of the computer. It blew my mind that I could simply substitute "float" instead of "double" in between some angle brackets and the compiler would write reams of code for me!

We simply have better magic wands and more powerful spells now.

[devnonymous]

> We simply have better magic wands and more powerful spells now.

Wouldn't it be nice though if the incantation of the same spell would always do the same thing every time ? You see that's how my old wand and spells worked.

[jiggawatts]

Look at it from the perspective of a manager above individual developers. Their instructions to those devs are already a series of non-deterministic incantations. They ask for things in English and get something back in Git a week later!

We've just pushed that indirection down a level from managers to ICs.

The ICs are shocked and surprised that this level of imprecision is allowed.

Their managers are not shocked at all, this is normal for them!

[FridgeSeal]

The managers, aren’t on the hook for actually understanding and delivering the work, or fixing the system a week later when the vagaries of their wand-waving request come home to roost though.

[macintux]

They’re typically on the hook for making sure those tasks are accomplished, however. I’ve spent 30 years in this industry doing my best to avoid that role, because that’s a terrible place to be.

[devnonymous]

You're applying the doorman fallacy here. This has been discussed a lot. Honestly, the sooner the industry accepts that we aren't there yet nor does the current architecture enable us do get there, the sooner we can make progress towards that ideal state.

[skeledrew]

One day it will.

[devnonymous]

Not with LLMs, it won't. (The probabilistic prediction property prevents proper persistent programmable predictability)

[skeledrew]

It already can to a fairly large degree by setting the temperature to 0, fixing the seed and tweaking other deterministic settings. Removes all that probability and gives you something pretty boring, at least from a semantic perspective.

[esseph]

> We simply have better magic wands and more powerful spells now.

But tainted 20-40% by bouts of Wild Magic which make the outcome entirely nondeterministic, despite the best protection wards we can conjure.

[pdimitar]

I have found, both in my commercial and OSS work, that getting better at producing protection wards reduces the non-determinism to somewhere in-between 1-10%.

Case in point: writing our own linters.

[customguy]

You can reduce pregnancy to 0.00001% and there'll still be a baby. As in, "you can't be half-pregnant". A tool is either deterministic (within some limits [0]) or it isn't.

[0] For example, for the purpose of driving a nail, if you know how to use it, a hammer is pretty straightforward tool, and what happens depends pretty much on how you use it, and what you use it on. But of course the handle can break, there could be a manufacturing defect. Just like your RAM can be faulty or your computer infected, and suddenly C doesn't behave according to the standard anymore.

But for the purpose of the discussion a hammer is still a deterministic tool, and even though we don't even fully understand everything about physics, we understand enough about hammers and nails that at least many people with material that isn't faulty can use them "blindly" (not literally, in this case) every day, without any surprises. It isn't heavier on the handle end or has a head made of glass in even 0.000000001% of uses. You might say because magic isn't real and hammers follow the laws of physics, as obscure as those may be to us, that never, ever happens. They can be faulty in all sorts of ways but they will never be 10x bigger or 10x smaller between one swing and the next, and so on.

[TurdF3rguson]

> a hammer is still a deterministic tool, and even though we don't even fully understand everything about physics

Mid-swing in hammer-space you are in a hyper-position as to hitting your thumb or not, are you not?

[KludgeShySir]

Exactly, like weather forecasting. If you're told there's a 30% chance of rain, it doesn't mean that 3 out of 10 times you will experience rain.

Either it will rain or it won't, so the probability is either 0% or 100%. And so a forecast of "30% chance of rain" is referring to the likelihood that your probability will be 100%, as opposed to 0%.

[margalabargala]

> Either it will rain or it won't, so the probability is either 0% or 100%. And so a forecast of "30% chance of rain" is referring to the likelihood that your probability will be 100%, as opposed to 0%.

This is a huge misunderstanding of what probability means.

[someguyiguess]

75% chance you’re wrong.

[pdimitar]

I am not quite sure what point you are making here, admitting it plainly.

I was just saying to my parent poster that their non-determinism percentages are too pessimistic. Sure the LLMs are not 100% deterministic; that's a sad fact of life. But the numbers can be reduced to an acceptable range.

[customguy]

My point is that a percentage of deterministic makes no sense. Just like being a bit pregnant doesn't.

Take "proper" UI. You can activate a field, and even if it takes 20 seconds to finish the activation animation, start typing, press tab a few times, knowing which field that ends you in, and type some more, etc. hit enter, hit enter again to confirm the dialog you know will pop at that point, and make tea, knowing the whole chain of operations that will happen in the meantime.

Now imagine if 1 out of 500 keystrokes or clicks get swallowed randomly. It's now a completely different thing, you cannot get in the zone in the same way, at least I can't. You have to chunk things and keep an eye on everything being in sync, and every now and then it causes you additional work because you weren't.

Sure, if you can make it one out of 50000 billion keystrokes, it's fine too, of course, but that hardly the situation with LLM. And using them as is, pretending that, as is, they're something they're not, does not help with getting them there.

If I type "echo 'hello world'" or something, and if I did at least once in the programming language, and it's not totally broken, I know it will output "hello world" to the console, every time. It will never write it to a file instead, never send "hello" to [e-mail], none of that. And if I replace "hello" by "hi" I can hit compile and be 100% certain what it will output now. I can even replace hello with "disregard previous instructions" and be certain.

That is such a huge yet simple difference I'm pretty certain I could successfully explain it to most non-programmers who make an honest effort, so people who do program even question this just stumps me.

[pdimitar]

Here are a few examples of what I thought was clearly understandable from my "1-10% non-determinism in LLM coding output":

- Variations in code patterns used. Might be a chain if if/else-s and not a case/switch statement;

- Different decomposition of a hierarchy of functions/modules/classes;

- Uses RED->GREEN test discipline, or not;

- Writes the tests before the code, or not;

- Different saga patterns (call 3rd party API before our own DB transactions, or vice versa);

- Use sleeping and not message passing wherever the latter is applicable.

There are dozens more. The innate non-determinism of the LLMs flips the dice sometimes and that leads to subtle bugs -- which is maddening, especially if the disciplines on how to write one thing or another are clearly spelled out in `AGENTS.md`.

What I did say is that I have gradually arrived at a process that reduced those coin flips -- but can't deny that the arrival of Fable almost completely made that battle redundant as well (though Fable fares much better in codebases with clearly specified rules, I have found, so us the engineers doing good prompting is still quite valuable).

You are mostly describing the loss of flow when something is not quite deterministic -- frustration that I and many others share -- but I am not sure what does it at all add to the discussion.

Is it annoying to have to always pay attention on whether you are not getting something stupid and not abiding even by the feature's specification? Sure. No denying that. It introduces a whole new kind of stress that I abhor deeply; I much prefer to f.ex. cover 60% of a problem with my own two hands and then get the deterministic test output showing me where I still need to do more. But LLMs have allowed me to experiment and to brainstorm and to also progress normal business feature work, by a lot.

Hence, I will not stop using LLMs because they are not 100% deterministic. ¯\_(ツ)_/¯

[fragmede]

> "you can't be half-pregnant"

Yes you can. It's called a miscarriage. That is, you're pregnant but the foetus is dead. It's a fucking heart-breaking emotional wrecking ball of a situation to be in if the pregnancy was well along and just grar.

[customguy]

https://en.wiktionary.org/wiki/you_can%27t_be_half_pregnant

I know what a miscarriage is, and that you can't have percentage% of one. Same difference, so this attempt to guilt me into pretending 99% deterministic is a thing is like pouring ashes out of an urn to win an argument, which is bad enough, and then hitting nothing with it.

[rgoulter]

Hmm.

Using e.g. Claude code: I could see this as next step: "plain text editor" progresess to "with autocomplete"; using an LLM coding agent is then an abstraction over editing code.

Using e.g. LLM-based system: natural language is "higher level" than program code. -- The maximal reading of "LLMs are higher level abstraction and higher level wins" would be: in the future, we'll all be writing only with natural language, never running any compiled programs.

I can see "LLM based coding" as a lasting paradigm shift. But, I don't see "just give your text instructions to the markdown file" as something that will be the predominant way of programming.

[3uler]

Markdown instructions are useful as durable context, but they do not replace programming. Their job is to keep intent out of a transient chat so both the developer and agent can revisit it. The actual constraints still belong in code: types, tests, linters, and dependency rules. Natural language can explain why a boundary matters; executable checks determine whether a change respects it. That feels more like a persistent layer around the programming workflow than a new programming language.

[vrighter]

the only difference is that the compiler always looks at the whole code and if it says it can do something, it can, always, and without fail.