[AlotOfReading]

The problem with the author's argument is the inductions don't follow from the premise. With defined C, you can in principle look at a piece of code and know what it will do in the abstract machine (or at least build a model dependent on assumptions about things like unspecified behavior). Actually doing this may be practically impossible, but that's not the point. It's not possible in the presence of UB. You can't know what a piece of code containing UB will do, even in principle.

You can in principle read the LLM's output and know that it won't put your credentials on the net, so it's not the same as UB. Maybe there are practical similarities to UB in how LLM bugs present, but I'm not sure it's a useful comparison and it's not the argument the author made.

[pseudohadamard]

The practical impossibility is a real issue, see the recent post about booleans in Doom where the author knew what the problem was, where it was, and what it was, but after reading through the standards still couldn't really find where in the standards it was forbidden, eventually saying "it's probably this bit because I can't find anything else that fits".

And when the author of the current post says:

  Turn on all linting, all warnings,

this doesn't help. I've seen code compiled with -Wall -Wextra -Wtf that produces zero warnings but for which gcc happily outputs code that segfaults, crashes, or otherwise breaks catastrophically when run. So the compiler is saying "I've found UB here, I'm not going to say anything despite maximum warnings being turned on, I'm just going to output code that I know will fail when run".

[lelanthran]

> The problem with the author's argument is the inductions don't follow from the premise.

That's possible. No one ever accused me of sound arguments :-)

I would still like to address your comment anyway.

Lets call this assertion #1:

> With defined C, you can in principle look at a piece of code and know what it will do in the abstract machine ... It's not possible in the presence of UB.

And lets call this assertion #2:

> You can in principle read the LLM's output and know that it won't put your credentials on the net, so it's not the same as UB.

With Assertion #1 you state you are not examining the output of the compiler, you are only examining the input (i.e. the source code).

With Assertion #2, you state you are examining the output of the LLM, and you are not examining the input.

IOW, these two actions are not comparable because in one you examine only the input while in the other you examine only the output.

In short: you are comparing analysing the input in one case with analysing the output in another case.

For the case of accidentally doing $FOO when trying to do $BAR:

1. No amount of input-analysis on LLM prompts will ever reveal to you if it generated code that will do $FOO - you have to analyse the output. There is a zero percent chance that examining the prompt "Do $BAR" will reveal to the examiner that their credentials will be leaked by the generated code.

2. There is a large number of automated input-analysis for C that will catch a large number of UB that prevents $FOO, when the code implements "Do $BAR". Additionally, while a lot of UB gets through, a great deal are actually caught during review.

Think of the case: "I wrote code to add two numbers, but UB caused files to get deleted off my computer"

In C, this was always possible (and C programmers acted accordingly). In Java, C#, Rust, etc this was never possible. Unless your code was generated by an LLM.

[AlotOfReading]

That's a good point, I didn't realize I was implicitly mixing up inputs and outputs.

I think you're imagining a very particular way of using LLMs though. The source code is the source of truth in traditional development. It's the artifact we preserve long term and the one that's used to regenerate ephemeral artifacts like binaries. When you regenerate binaries from source code containing UB, the result may not behave the same as before. Each binary's semantics can be individually understood, but not the semantics of future translations.

If you treat the entire LLM->binary system as a black box, then yeah. I agree there's no reasonable way to go from input to output semantics, much as there isn't if you ask a human. But people generally aren't using the prompt as the source of truth. They're using the code that's produced, which (in the absence of traditional UB) will have the same semantics every time it's used even if the initial LLM doesn't.