[quelsolaar]

The 5 stages of learning about UB in C:

-Denial: "I know what signed overflow does on my machine."

-Anger: "This compiler is trash! why doesn't it just do what I say!?"

-Bargaining: "I'm submitting this proposal to wg14 to fix C..."

-Depression: "Can you rely on C code for anything?"

-Acceptance: "Just dont write UB."

[matheusmoreira]

What stage is the "just make the compiler define the undefined" stage?

Unaligned access? Packed structs. Compiler will magically generate the correct code, as if it had always known how to do it right all along! Because it has, in fact, always known how to do it right. It just didn't.

Strict aliasing? Union type punning. Literally documented to work in any compiler that matters, despite the holy C standard never saying so. Alternatively, just disable it straight up: -fno-strict-aliasing. Enjoy reinterpreting memory as you see fit. You might hit some sharp edges here and there but they sure as hell aren't gonna be coming from the compiler.

Overflow? Just make it defined: -fwrapv. Replace +, -, * with __builtin_*_overflow while you're at it, and you even get explicit error checking for free. Nice functional interface. Generates efficient code too.

The "acceptance" stage is really "nobody sane actually cares about the C standard". The standard is garbage, only the compilers matter. And it turns out that compilers have plenty of extremely useful functions that let you side step most if not all of this. People just don't use this because they want to write "portable" "standard" C. The real acceptance is to break out of that mindset.

Somehow I built an entire lisp interpreter in freestanding C that actually managed to pass UBSan just by following the above logic. I was actually surprised at first: I expected it to crash and burn, but it didn't. So if I can do it, then anyone can do it too.

[quelsolaar]

A lot of the Central UB can not be defined, because they rely on detection. In order to have a well defined behaviour (by the standard or the compiler) the implementation needs to first detect that the behaviour is triggered, this is often very tricky or expensive. Its easy to define that a program should halt, if it writes outside an array, but detecting if it does can be both slow and hard to implement. There are implementations that do, but they are rarely used outside of debugging.

A better way to think about UB is as a contract between developer and implementation, so that the implementations can more easily reason about the code. How would you optimize:

(x * 2) / 2

An optimizer can optimize this out for a signed integer, because it doesn't have to consider overflow, but with a unsigned integer it can not. UB is a big reason why C is the most power efficient high level language.

[matheusmoreira]

> How would you optimize: (x * 2) / 2

I'd do the math myself and just write x.

I don't even use * for multiplication anymore, I use __builtin_mul_overflow and then check the result. Anyone who doesn't is gonna hit the overflow case one day, and they'll be lucky if their program isn't exploited because of it. I've been making an effort to use all the overflow checking builtins by default in most if not all cases. I've also been making Claude audit every single bare arithmetic operation in my projects. He's caught quite a few security issues already, and overflow checking dealt with them all.

This particular contract between developer and implementation is totally worthless and doing more harm than good. It encompasses regular everyday normal things like multiplication and addition. All things that our brains literally rely on in order to reason about the code. Can't even add numbers without the compiler screwing it up.

Programmers need to deal with overflow at all times. Can't calculate an offset without dealing with overflow. Can't calculate a size without dealing with overflow. It's simply everywhere in systems programming, which is what C was designed to do. The consequence of ignoring this is usually that your program gets mercilessly exploited.

All this for some efficiency gains. The cost/benefit analysis is way off here. Things should be correct, first and foremost. Then the compiler should give us the necessary sharp tools to make it fast, if needed. It shouldn't be making it fast at the cost of turning the entire language into a memetic vulnerability machine.

[pjmlp]

The thing with (x * 2) / 2 is that for all practical purposes you might even have written something else, so the expression cannot be replaced by x directly.

What happens is that after a few common expression eliminations, peephole optmisations, code inlining, and possibly other optimisation passes, the remaining AST will be (x * 2) / 2, and then the magic happens.

[matheusmoreira]

That makes sense... I agree, and I don't really have an elegant answer to that at the moment.

I simply accepted the fact that the magic might stop happening at (x * 2) / 2. This acceptance bought me certainty about overflow behavior. I think it was a good tradeoff.

[quelsolaar]

The things you want from C isn't C. Id advice you to use another language.

[matheusmoreira]

No. I like C. I've learned about a dozen languages by now. I always end up coming back to C. I've just accepted it.

There is no reason whatsoever that C can't be improved. Compiler attributes and builtins are already doing quite a lot of heavy lifting. Recent addition: counted_by, an attribute that allows compilers to properly track the size of memory referenced by pointers. All C programmers should be making liberal use of this stuff.

[gpderetta]

> Unaligned access? Packed structs.

Packed structs are dangerous. You can do unaligned accesses through a packed type, but once you take the address of your misaligned int field, then you are back into UB territory. Very annoying in C++ when you try to pass the a misaligned field through what happens to be generic code that takes a const reference, as it will trigger a compiler warning. Unary operator+ is your friend.

[matheusmoreira]

> but once you take the address of your misaligned int field

Gotta work with the structure directly by taking the address of the packed structure itself.

  struct uu64 {
      u64 value;
  } __attribute__((packed));

  struct uu64 unaligned;
  struct uu64 *address = &unaligned;

  address->value; // this works

  u64 *broken = &address->value; // this doesn't

Taking the address of the field inside the structure essentially casts away the alignment information that was explicitly added to stop the compiler from screwing things up. So it should not be done.

Mercifully, both gcc and clang emit address-of-packed-member warnings if it's done. So the packed structures are effectively turning silently broken nonsense code into sensible warnings. Major win.

[Georgelemental]

> Strict aliasing? Union type punning. Literally documented to work in any compiler that matters, despite the holy C standard never saying so.

It does say so, actually, since C99 TC3 (DR 283).

[matheusmoreira]

Thanks, I stand corrected.

[duped]

> People just don't use this because they want to write "portable" "standard" C

Something that bothers me is the Venn diagram of people that think abstraction is slow and error prone and people that only write portable C.

How many C implementations do you actually need to compile against? I don't think I've seen more than 3 outside Unix software from the 90s. Using non portable extensions is in fact totally doable for your application and you should probably do it, and just duplicate/triplicate code where you have to. It's not that hard to write and not hard to read.

[pjmlp]

Back when I still wrote C at work, it meant Aix xlC, HP-UX aCC, Solaris Forte, Red-Hat Linux GCC, Windows MSVC and C++ Builder.

Nowadays most are indeed clang and GCC forks, or MSVC.

[duped]

That's what I mean, I've seen enough autoconf "checking for <feature that totally works in every compiler you care about>" noise to know it's mostly pointless in this day in age.

[lelanthran]

> What stage is the "just make the compiler define the undefined" stage?

It can be left as implementation defined, which means that the compiler can't simply do arbitrary things, it needs to document what it would do.

Take, for example, signed-integer overflow: currently a compiler can simply refuse to emit the code in one spot while emitting it in another spot in the same compilation unit! Making it IB means that the compiler vendor will be forced to define what happens when a signed-integer overflows, rather than just saying, as they do now, "you cannot do that, and if you do we can ignore it, correct it, replace it or simply travel back in time and corrupt your program".

> Somehow I built an entire lisp interpreter in freestanding C that actually managed to pass UBSan just by following the above logic. I was actually surprised at first: I expected it to crash and burn, but it didn't. So if I can do it, then anyone can do it too.

Same here; I built a few non-trivial things that passed the first attempt at tooling (valgrind, UBsan with tests, fuzzing, etc) with no UB issues found.

[matheusmoreira]

Completely agree. It can, and I think it's extremely annoying that it wasn't.

So we have the next best thing: builtins and flags. So long as those cover all the undefined behavior there is, we can live with it. Compiler gets to be "conformant" and we get to do useful things without the compiler folding the code into itself and inside out.

[MrBuddyCasino]

One of these days you should write a blogpost about this.

[pjmlp]

Lost in the submission attempt to WG14.

[thomashabets2]

Author here.

> -Acceptance: "Just dont write UB."

The point of my article is that this is not possible. This cannot be our end state, as long as humans are the ones writing the code. No human can avoid writing UB in C/C++.

[jart]

It's honestly not that difficult to be rigorous. The things you mentioned in the blog post are pretty obvious forms of degenerate practices once you get used to seeing them. The best way to make your argument would be to bring up pointer overflow being ub. What's great about undefined behavior is that the C language doesn't require you to care. You can play fast and loose as much as you want. You can even use implicit types and yolo your app, writing C that more closely resembles JavaScript, just like how traditional k&r c devs did back in the day under an ilp32 model. Then you add the rigor later if you care about it. For most stuff, like an experiment, we obviously don't care, but when I do, I can usually one shot a file without any UB (which I check by reading the assembly output after building it with UBSAN) except there's just one thing that I usually can't eliminate, which is the compiler generating code that checks for pointer overflow. Because that's just such a ridiculous concept on modern machines which have a 56 bit address space. Maybe it mattered when coding for platforms like i8086. I've seen almost no code that cares about this. I have to sometimes, in my C library. It's important that functions like memchr() for example don't say `for (char *p = data, *e = data + size; p<e; ...` and instead say `for (size_t i = 0; i < n; ++i) ...data[i]...`. But these are just the skills you get with mastery, which is what makes it fun. Oh speaking of which, another fun thing everyone misses is the pitfalls of vectorization. You have to venture off into UB land in order to get better performance. But readahead can get you into trouble if you're trying to scan something like a string that's at the end of a memory page, where the subsequent page isn't mapped. My other favorite thing is designing code in such a way that the stack frame of any given function never exceeds 4096 bytes, and using alloca in a bounded way that pokes pages if it must be exceeded. If you want to have a fun time experiencing why the trickiness of UB rules are the way they are, try writing your own malloc() function that uses shorts and having it be on the stack, so you can have dynamic memory in a signal handler.

[thomashabets2]

> It's honestly not that difficult to be rigorous.

Ok, let's try it. I pointed GPT 5.5 at the smallest part of cosmopolitan as I could find in two seconds, net/finger. 299 lines.

describesyn.c:66: q + 13 constructs a pointer that can point well beyond the array plus one element.

C23 6.5.6p9:

> If the pointer operand and the result do not point to elements of the same array object or one past the last element of the array object, the behavior is undefined

Now… you may be trolling, but I do feel like this disproves your assertion. Not you, not me, not Theo de Raadt, can avoid UB.

> the compiler generating code that checks for pointer overflow.

Do you need to check for that specifically? What pointer are you constructing that is not either pointing at a valid object correctly aligned (not UB), or exactly one past the element of an array?

Do you mean for the latter, in case you have an array that ends on the maximum expressible pointer address?

I'm a bit unclear on what you mean by "pointer overflow". From mentioning 56 bit address spaces I'm guessing you mean like the pointer wrapped, not what I pointed to in cosmopolitan, above?

Ok, to be clear that it's not just that one type, if you forgive that one:

net/http/base32.c:64: read sc[0] even if sl=0. I assume this is never called with sl=0, so could be fine.

net/http/ssh.c:355: pointer address underflow? Should that be `e - lp`?

net/http/ssh.c:209/229: double destroy of key. can this code path have non-null members, meaning double free? Looks like it, since line 207 does the parsing and checks that parse worked.

net/http/ssh.c:123: uses memset, which assumes that it sets member variable pointers to NULL (per my post, depending on that means depending on UB), and later these pointers are given to free(), so that's UB.

I won't look deeper into net/http, but presenting just the possibly incorrect remaining comments from jippity:

  - ssh.c:211 and parsecidr.c:44: length-taking APIs use unbounded strstr() / strchr(), so explicit n with non-NUL-terminated input can read beyond the buffer.

  - tokenbucket.c:77 and tokenbucket.c:92: x >> (32 - c) is UB for c == 0 and for out-of-range c.

  - isacceptablehost.c:68: long numeric host labels can overflow signed int b before the function eventually rejects/accepts the host.

[andai]

> For most stuff, like an experiment, we obviously don't care, but when I do, I can usually one shot a file without any UB (which I check by reading the assembly output after building it with UBSAN)

Does this depend on the project, or part of a project? I'm wondering how far that scales, I don't know labor intensive it is -- maybe you can just look at the output and see that nothing funny is happening?

[frollogaston]

"Just don't write UB" sounds like still part of the bargaining stage at best

[im3w1l]

In C, acceptance is "I will write UB and it will eventually lead to something bad happening"

[superxpro12]

Just work on embedded devices like I did lol. It's so nice to write software targeting a specific cpu.

[Ygg2]

> -Acceptance: "Just dont write UB."

Just switch to a saner language.

And before I get attacked for being a Rust shill, I meant Java :P

The bar is so low it's floating near the center of the Earth.

[dns_snek]

> And before I get attacked for being a Rust shill, I meant Java :P

If all you want is C but less insane then the obvious answer here is Zig.

[simonask]

Zig is cool, but it is not even close to being ready for prime-time. It will be pre-1.0 for a while, and major breaking changes are still happening.

[dns_snek]

Sure, maybe don't bet your entire company on mountains of Zig code just yet, but aside from the breaking changes it's been perfectly usable and suitable for every project I've ever wanted to work on.

[psychoslave]

If all somebody want is a programming language than C/C++ on these matter, there are plentiful options of the shelf to pick from.

If all somebody want is a turn key replacement to C/C++ ecosystem, then there is nothing like that in the world that I’m aware of.

[AgentME]

If someone is switching from C because it's too easy to trigger undefined behavior, picking one of the few other not memory safe languages is missing the point.

[dns_snek]

>> If all you want is C but less insane

Zig has orders of magnitude less undefined behavior and orders of magnitude more memory safety than C. It's not a binary situation.

[pjmlp]

Object Pascal, with 40 years of experience, no need to wait for 1.0.

[p2detar]

> Just switch to a saner language.

And where's the fun in that?

[psychoslave]

That’s a taste matter. Being recalled that what is expressed is always depending on some technical details on every move, this is great when one is loving technical details and have all the leisure time to pay attention to them. This is going to be hell compared to sound defaults for someone willing to focus on delivering higher order feature/functionality which will most likely work just fine.

Unedefined behaviour means "we couldn’t settle on a best default trade-off with fine-tuning as a given option so we let everyone in the unknown".

[ErroneousBosh]

Okay, so Java compiles to machine code now?

Because the last time I looked it appeared to need some godawful slow bytecode interpreter that took up thousands of kilobytes of RAM.

[elch]

If you don't like JIT/JVM there's GraalVM Native Image.

https://www.graalvm.org/latest/reference-manual/native-image...

In the past you could use e.g. Excelsior JET.

[ErroneousBosh]

Great, can you fit it into 768 bytes of flash and 64 bytes of RAM?

[crote]

It isn't 1970 anymore. You can get 32-bit ARM MCUs with tens of kilobytes of flash and multiple kilobytes of RAM for less than 10 cents.

We've long since reached a point where chips are cheap enough to be disposable. They are included in paper transit tickets and price tags. There is basically no market left where your volume is small enough that custom application-specific ICs aren't an option, but your volume is large enough that the cost of a few additional kilobytes of memory isn't massively outweighed by the developer time saved.

Want several megabytes of RAM and flash to run Java? That's the price of a cup of coffee!

[ErroneousBosh]

> It isn't 1970 anymore. You can get 32-bit ARM MCUs with tens of kilobytes of flash and multiple kilobytes of RAM for less than 10 cents.

Do they run at single-digit nA current draw?

[blacklion]

> Because the last time I looked it appeared to need some godawful slow bytecode interpreter that took up thousands of kilobytes of RAM.

Did you looked at java 1.2 at 1998 last time? Because after that there is compiler which produce some very efficient profile-guide-optimized code and do tricks like de-virtualization which is not possible with static compiler with support of multiple compilation units (like C++).

Really, there was time in history when HotSpot-compiled JVM bytecode was faster than everything that gcc could produce for comparable tasks. Yes, now this gap is reversed again, as both gcc and clang become much more clever, but still gap is not very wide now.

[pjmlp]

For 26 years already, it is a matter of choosing the right JDK.

[pjc50]

Java has been jitted for .. decades?

[Hendrikto]

You know what JIT means, right? It means that is is not compiled from the start and indeed runs on a bytecode interpreter until the JIT compiler kicks in.

[fc417fc802]

The java JIT has produced sufficiently fast code for all but the most demanding of HPC applications for going on 20 years. I realize keeping up with new developments can be difficult but the out of date java performance memes are entirely ridiculous by now.

Meanwhile half the world appears to run on cpython of all things.

[agentultra]

My life for a browser that doesn't jitter and tear when scrolling or a terminal emulator that can actually process data near the speed my hardware can handle.

[Hendrikto]

Yes, the JIT compiler compiles code. Yes, the results are good. That does not change the fact that the JVM still has and uses a bytecode interpreter, which the comment I replied to disputed.

[1718627440]

> -Denial: "I know what signed overflow does on my machine."

Or you just not skip the introductory pages, that tell you what the language philosophy of C is, and why there is UB. Yes, UB can be a struggle, but the first four steps are entirely unnecessary. It means that you do not actually understand the core concepts of the very same language you are using, which is kinda stupid.

[whizzter]

I think the issue has been that the line between de-jure and de-facto behaviours has shifted over the years as compiler optimizations suddenly began relying on de-jure intrepretations of UB to increase performance while ignoring de-facto usage of the language.

When that started happened people became alarmed (oMG UB iS TeH BAD!) and since some old UB machines still had industry support (of organisations that actually participated in ISO meetings instead of arguing online) there was never any movement on defining de-facto usage as de-jure and the alarmist position became the default.

Personally I think the industry would've benefited from a Boring C (as described by DJB) push by people that would've created a public parallell "de-jure" standard that would've had a chance to be adopted by compiler creators.

[1718627440]

> I think the issue has been that the line between de-jure and de-facto behaviours has shifted over the years as compiler optimizations suddenly began relying on de-jure intrepretations of UB to increase performance while ignoring de-facto usage of the language.

I guess I am too young, and also too much a purist, because I start from the impression of what the language is, not what the implementations happen to do.

> Personally I think the industry would've benefited from a Boring C (as described by DJB) push by people that would've created a public parallell "de-jure" standard that would've had a chance to be adopted by compiler creators.

-O0