One has to add that from the 218 UB in the ISO C23, 87 are in the core language. From those we already removed 26 and are in progress of removing many others. You can find my latest update here (since then there was also some progress): https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3529.pdf
A lot of that work is basically fixing documentation bugs, labelled "ghosts" in your text. Places where the ISO document is so bad as a description of C that you would think there's Undefined Behaviour but it's actually just poorly written.
Fixing the document is worthwhile, and certainly a reminder that WG21's equivalent effort needs to make the list before it can even begin that process on its even longer document, but practical C programmers don't read the document and since this UB was a "ghost" they weren't tripped by it. Removing items from the list this way does not translate to the meaningful safety improvement you might imagine.
There's not a whole lot of movement there towards actually fixing the problem. Maybe it will come later?
> practical C programmers don't read the document and since this UB was a "ghost" they weren't tripped by it
I would strongly suspect that C compiler implementers very much do read the document, though. Which, as far as I can see, means "ghosts" could easily become actual UB (and worse, sneaky UB that you wouldn't expect.)
The previous language might cause a C compiler developer to get very confused because it seems as though they can choose something else but what it is isn't specified, but almost invariably eventually they'll realise oh, it's just badly worded and didn't mean "should" there.
It's like one of those tricky self-referential parlor box statements. "The statement on this box is not true"? Thanks I guess. But that's a game, the puzzles are supposed to be like that, whereas the mission of the ISO document was not to confuse people, so it's good that it is being improved.
Most of the "ghosts" are indeed just cleaning up the wording. But compiler writers historically often used any excuse that the standard is not clear to justify aggressive optimization. This starts with an overreaching interpretation of UB itself, to wacky concepts such as time-travel, wobbly numbers, incorrect implementation of aliasing (e.g. still in clang), and pointer-to-integer round trips.
I'm sure the compiler authors will disagree that they were "using any excuse". From their point of view they were merely making transformations between equivalent programs, and so any mistake is either that these are not in fact equivalent programs because they screwed up - which is certainly sometimes the case - or the standard should not have said they were equivalent but it did.
One huge thing they have on their side is that their implementation is concrete. Whatever it is that, say, GCC does is de facto actually a thing a compiler can do. The standards bodies (and WG21 has been worse by some margin, but they're both guilty) may standardize anything, but concretely the compiler can only implement some things. "Just do X" where X isn't practical works fine on paper but is not implementable. This was the fate of the Consume ordering. Consume/ Release works fine on paper, you "just" need to have whole program analysis to implement it. Well of course that's not practical so it's not implemented.
If I understand correctly, the "ghosts" are vacuously UB. As in, the standard specifies that if X, then UB, but X can in fact never be true according to the standard.
And yet, I see P1434R0 seemingly trying to introduce new undefined behavior, around integer-to-pointer conversions, where previously you had reasonably sensible implementation defined behavior (the conversions “are intended to be consistent with the addressing structure of the execution environment").
Pointer provenance already existed before, but the standards were contradictory and incomplete. This is an effort to more rigorously nail down the semantics.
i.e., the UB already existed, but it was not explicit had to be inferred from the whole text and the boundaries were fuzzy. Remember that anything not explicitly defined by the standard, is implicitly undefined.
Also remember, just because you can legally construct a pointer it doesn't mean it is safe to dereference.
The current standard still says integer-to-pointer conversions are implementation defined (not undefined) and furthermore "intended to be consistent with the addressing structure of the execution environment" (that's a direct quote).
I have an execution environment, Wasm, where doing this is pretty well defined, in fact. So if I want to read the memory at address 12345, which is within bounds of the linear memory (and there's a builtin to make sure), why should it be undefined behavior?
And regarding pointer provenance, why should going through a pointer-to-integer and integer-to-pointer conversions try to preserve provenance at all, and be undefined behavior in situations where that provenance is ambiguous?
The reason I'm using integer (rather than pointer) arithmetic is precisely so I don't have to be bound by pointer arithmetic rules. What good purpose does it serve for this to be undefined (rather than implementation defined) beyond preventing certain programs to be meaningfully written at all?
I fully agree with your analysis but compilers writers did think the could bend the rules, hence it was necessary to clarify that pointer-to-integer casts do work as intended. This still not in ISO C 23 btw because some compiler vendors did argue against it. But it is a TS now. If you are, please file bugs against your compilers.
Do you fully agree? I finally went and read n3005.pdf. The important item there is that a cast to integer exposes the pointer and now the compiler must be conservative and assume that the pointed object might be changed via non trackable pointers. This seems quite a reasonable compromise to make existing code work without affecting the vast majority of objects whose address is never cast to an integer. But ncruces wants defined semantics for arbitrary forged pointers.
> I have an execution environment, Wasm, where doing this is pretty well defined, in fact. So if I want to read the memory at address 12345, which is within bounds of the linear memory (and there's a builtin to make sure), why should it be undefined behavior?
How would you define it? Especially in a way that is consistent with the rest of the language and allows common optimizations (remember that C supports variables, which may or may not be stored in memory)?
Just read whatever is at address 12345 of the linear memory. Doesn't matter what that is. If it's an object, if it was malloc'ed, if it's the "C stack", a "global".
It's the only way to interpret *(uint64_t*)(12345) when the standard says that a integer-to-pointer conversion is "intended to be consistent with the addressing structure of the execution environment".
There exists an instruction to do that load in Wasm, there's a builtin to check that 12345 points to addressable memory, the load is valid at the assembly level, the standard says the implementation should define this to be consistent with the addressing structure of the execution environment, why the heck are we playing games and allowing the compiler to say, "nope, that's not valid, so your entire program is invalid, and we can do what ever we want, no diagnostic required"?
It is important to understand why undefined behaviour has proliferated over the past ~25 years. Compiler developers are (like the rest of us) under pressure to improve metrics like the performance of compiled code. Often enough that's because a CPU vendor is the one paying for the work and has a particular target they need to reach at time of product launch, or there's a new optimization being implemented that has to be justified as showing a benefit on existing code.
The performance of compilers is frequently measured using the SPEC series of CPU benchmarks, and one of the main constraints of the series SPEC series of tests is that the source code of the benchmark cannot be changed. It is static.
As a result, compiler authors have to find increasingly convoluted ways to make it possible for various new compiler optimizations to be applied to the legacy code used in SPEC. Take 403.gcc: it's based on gcc version 3.2 which was released on August 14th 2002 -- nearly 23 years ago.
By making certain code patterns undefined behaviour, compiler developers are able to relax the constraints and allow various optimizations to be applied to legacy code in places which would not otherwise be possible. I believe the gcc optimization to eliminate NULL pointer checks when the pointer is dereferenced was motivated by such a scenario.
In the real world code tends to get updated when compilers are updated, or when performance optimizations are made, so there is no need for excessive compiler "heroics" to weasel its way into making optimizations apply via undefined behaviour. So long as SPEC is used to measure compiler performance using static and unchanging legacy code, we will continue to see compiler developers committing undefined behaviour madness.
The only way around this is for non-compiler developer folks to force language standards to prevent compilers from using undefined behaviour to do that which normal software developers considers to be utterly insane code transformations.
Language standards have much less power than people think and compiler-vendors are of course present in the standard working groups. Ultimately, the users need to put pressure on the compiler vendors. Please file bugs - even if this often has no effect, it takes away the argument "this is what our users want". Also please support compilers based on how they deal with UB and not on the latest benchmark posted somewhere.
In a compiler, you essentially need the ability to trace all the uses of an address, at least in the easy cases. Converting a pointer to an integer (or vice versa) isn't really a deal-breaker; it's essentially the same thing as passing (or receiving) a pointer to an unknown external function: the pointer escapes, whelp, nothing more we can do in that case for the most part.
But converting an integer to a pointer creates a problem if you allow that pointer to point to anything--it breaks all of the optimizations that assumed they could trace all of the uses of an address. So you need something like provenance to say that certain back-conversions are illegal. The most permissive model is a no-address-taken model (you can't forge a pointer to a variable whose address was never taken). But most compilers opt instead for a data-dependency-based model: essentially, even integer-based arithmetic of addresses aren't allowed to violate out-of-bounds at the point of dereference. Or at least, they claim to--the documentation for both gcc and llvm have this claim, but both have miscompilation bugs because they don't actually allow this.
The proposal for pointer provenance in C essentially looks at how compilers generally implement things and suggests a model that's closer to their actual implementation: pointer-to-integer exposes the address such that any integer-to-pointer can point to it. Note this is more permissive than the claimed models of compilers today--you're explicitly able to violate out-of-bounds rules here, so long as both objects have had their addresses exposed. There's some resistance to this because adhering to this model also breaks other optimizations (for example, (void*)(uintptr_t)x is not the same as x).
As a practical matter, pointer provenance isn't that big of a deal. It's not hard to come up with examples that illustrate behaviors that cause miscompilation or are undefined specifically because of pointer provenance. But I'm not aware of any application code that was actually miscompiled because the compiler implemented its provenance model incorrectly. The issue gets trickier as you move into systems code that exists somewhat outside the C object model, but even then, most of the relevant code can ignore their living outside the object model since resulting miscompiles are prevented by inherent optimization barriers anyways (note that to get a miscompile, you generally have to simultaneously forge the object's address, have the object's address be known to the compiler already, and have the compiler think the object's address wasn't exposed by other means).
Pointer provenance was certainly not here in the 80s. That's a more modern creation seeking to extract better performance from some applications at a cost of making others broken/unimplementable.
It's not something that exists in the hardware. It's also not a good idea, though trying to steer people away from it proved beyond my politics.
Pointer provenance probably dates back to the 70s, although not under that name.
The essential idea of pointer provenance is that it is somehow possible to enumerate all of the uses of a memory location (in a potentially very limited scope). By the time you need to introduce something like "volatile" to indicate to the compiler that there are unknown uses of a variable, you have to concede the point that the compiler needs to be able to track all the known uses within a compiler--and that process, of figuring out known uses, is pointer provenance.
As for optimizations, the primary optimization impacted by pointer provenance is... moving variables from stack memory to registers. It's basically a prerequisite for doing any optimization.
The thing is that traditionally, the pointer provenance model of compilers is generally a hand-wavey "trace dataflow back to the object address's source", which breaks down in that optimizers haven't maintained source-level data dependency for a few decades now. This hasn't been much of a problem in practice, because breaking data dependencies largely requires you to have pointers that have the same address, and you don't really run into a situation where you have two objects at the same address and you're playing around with pointers to their objects in a way that might cause the compiler to break the dependency, at least outside of contrived examples.
My grievance isn't with aliasing or dataflow, it's with a pointer provenance model which makes assumptions which are inconsistent with reality, optimises based on it, then justifies the nonsense that results with UB.
When the hardware behaviour and the pointer provenance model disagree, one should change the model, not change the behavior of the program.
This is sort of on the one hand not a meaningful claim, and then on the other hand not even really true if you squint anyway?
Firstly the hardware does not have pointers. It has addresses, and those really are integers. Rust's addr() method on pointers gets you just an address, for whatever that's worth to you, you could write it to a log maybe if you like ?
But the Morello hardware demonstrates CHERI, an ARM feature in which a pointer has some associated information that's not the address, a sort of hardware provenance.
I'm not a compiler writer, but I don't know how you would be able to implement any optimization while allowing arbitrary pointer forging and without whole-program analysis.
Say you're working with assembly as your medium, on a von neumann machine. Writing to parts of the code section is expected behaviour. What can you optimise in such a world? Whatever cannot be observed. Which might mean replacing instructions with sequences of the same length, or it might mean you can't work out anything at all.
C is much more restricted. The "function code" isn't there, forging pointers to the middle of a function is not a thing, nor is writing to one to change the function. Thus the dataflow is much easier, be a little careful with addresses of starts of functions and you're good.
Likewise the stack pointer is hidden - you can't index into the caller's frame - so the compiler is free to choose where to put things. You can't even index into your own frame so any variable whose address is not taken can go into a register with no further thought.
That's the point of higher level languages, broadly. You rule out forms of introspection, which allows more stuff to change.
C++ has taken this too far with the object model in my opinion but the committee disagrees.
Why? What specific optimization do you have in mind that prevents me from doing an aligned 16/32/64-byte vector load that covers the address pointed to by a valid char*?
It very much is something that exists in hardware. One of the major reasons why people finally discovered the provenance UB lurking in the standard is because of the CHERI architecture.
So it's something that exists in some hardware. Are you claiming that it exists in all hardware, and we only realized that because of CHERI? Or are you claiming that it exists in CHERI hardware, but not in others.
If it only exists in some hardware, how should the standard deal with that?
Undefined behavior only means that ISO C doesn't give requirements, not that nobody gives requirements. Many useful extensions are instances where undefined behavior is documented by an implementation.
Including a header that is not in the program, and not in ISO C, is undefined behavior. So is calling a function that is not in ISO C and not in the program. (If the function is not anywhere, the program won't link. But if it is somewhere, then ISO C has nothing to say about its behavior.)
Correct, portable POSIX C programs have undefined behavior in ISO C; only if we interpret them via IEEE 1003 are they defined by that document.
If you invent a new platform with a C compiler, you can have it such that #include <windows.h> reformats all the attached storage devices. ISO C allows this because it doesn't specify what happens if #include <windows.h> successfully resolves to a file and includes its contents. Those contents could be anything, including some compile-time instruction to do harm.
Even if a compiler's documentationd doesn't grant that a certain instance of undefined behavior is a documented extension, the existence of a de facto extension can be inferred empirically through numerous experiments: compiling test code and reverse engineering the object code.
Moreover, the source code for a compiler may be available; the behavior of something can be inferred from studying the code. The code could change in the next version. But so could the documentation; documentation can take away a documented extension the same way as a compiler code change can take away a de facto extension.
Speaking of object code: if you follow a programming paradigm of verifying the object code, then undefined behavior becomes moot, to an extent. You don't trust the compiler anyway. If the machine code has the behavior which implements the requirements that your project expects of the source code, then the necessary thing has been somehow obtained.
> Undefined behavior only means that ISO C doesn't give requirements, not that nobody gives requirements. Many useful extensions are instances where undefined behavior is documented by an implementation.
True, most compilers have sane defaults in many cases for things that are technically undefined (like take sizeof(void) or do pointer arithmetic on something other than a char). But not all of these cases can be saved by sane defaults.
Undefined behavior means the compiler can replace the code with whatever. So if you e.g. compile optimizing for size, the compiler will rip out the offending code, as replacing it with nothing yields the greatest size optimization.
Snippets of software exhibiting undefined behavior, executing e.g. both the true and the false branch of an if-statement or none etc. UB should not be taken lightly IMO...
> [...] undefined behavior, executing e.g. both the true and the false branch of an if-statement or none etc.
Or replacing all you mp3s with a Rick Roll. Technically legal.
(Some old version of GHC had a hilarious bug where it would delete any source code with a compiler error in it. Something like this would technically legal for most compiler errors a C compiler could spot.)
Unfortunely it also means that when the programmer fails to understand what undefined behaviour is exposed on their code, the compiler is free to take advantage of that to do the ultimate performance optimizations as means to beat compiler benchmarks.
The code change might come in something as innocent as a bug fix to the compiler.
Ah yes, the good old "compiler writers only care about benchmarks and are out to hurt everyone else" nonsense.
I for one am glad that compilers can assume that things that can't happen according to the language do in fact not happen and don't bloat my programs with code to handle them.
> I for one am glad that compilers can assume that things that can't happen according to the language do in fact not happen and don't bloat my programs with code to handle them.
Yes, unthinkable happenstances like addition on fixed-width integers overflowing! According to the language, signed integers can't overflow, so code like the following:
int new_offset = current_offset + 16;
if (new_offset < current_offset)
return -1; // Addition overflowed, something's wrong
can be optimized to the much leaner
int new_offset = current_offset + 16;
Well, I sure am glad the compiler helpfully reduced the bloat in my program!
You're objectively wrong. This code isn't bad, it's concise and fast (even without the compiler pattern-matching it to whatever overflow-detecting machine instructions happen to be available), and it would be valid and idiomatic for unsigned int. Stop blaming the code for your bad language spec.
Moral hazard here. The rest of us, and all of society, now rests on a huge pile of code written by incorrigible misers who imagined themselves able to write perfect, bug-free code that would go infinitely fast because bad things never happen. But see, there's bugs in your code and other people pay the cost.
Ransomware attacks against hospitals and a dark extortion economy churning tens if not hundreds of billions of dollars a year in losses and waste.
What would the "sky falling" look like to you? If you're expecting dramatic movie scenes like something out of Mr Robot, I'm afraid the reality is more mundane, just a never-ending series of basic programming errors that turn into remote code execution exploits because of language and compiler choices by people who don't pay the costs.
There's bugs in your code without undefined behavior too. Go use a different language if you don't care about performance, there are many to choose from.
Not only do I care about performance, the languages I use, are able to delivery both safety and performace at the level required for project delivery.
Unfortunely too many folks still pretend C is some kind of magic portable Assembly language that no other language on Earth is able to achieve the same.
Also if I care enough about ultimate performace, like anyone that actually cares about performance, I dust off my Assembly programming skills, alongside algorithms, datastructures and computer organisation.
> Including a header that is not in the program, and not in ISO C, is undefined behavior.
What is this supposed to mean? I can't think of any interpretation that makes sense.
I think ISO C defines the executable program to be something like the compiled translation units linked together. But header files do not have to have any particular correspondence to translation units. For example, a header might declare functions whose definitions are spread across multiple translation units, or define things that don't need any definitions in particular translation units (e.g. enum or struct definitions). It could even play macro tricks which means it declares or defines different things each time you include it.
Maybe you mean it's undefined behaviour to include a header file that declares functions that are not defined in any translation unit. I'm not sure even that is true, so long as you don't use those functions. It's definitely not true in C++, where it's only a problem (not sure if it's undefined exactly) if you ODR-rule use a function that has been declared but not defined anywhere. (Examples of ODR-rule use are calling or taking the address of the function, but not, for example, using sizeof on an expression that includes it.)
> I can't think of any interpretation that makes sense
Start with a concrete example. A header that is not in our program, or described in ISO C. How about:
#include <winkle.h>
Defined behavior or not? How can an implementation respond to this #include while remaining conforming? What are the limits on that response?
> But header files do not have to have any particular correspondence to translation units.
A header inclusion is just a mechanism that brings preprocessor tokens into a translation unit. So, what does the standard tell us about the tokens coming from #include <winkle.h> into whatever translation unit we put it into?
Say we have a single file program and we made that the first line. Without that include, it's a standard-conforming Hello World.
I think we are slowly getting closer to the crux of the matter. Are you saying that it's a problem to include files from a library since they are "not in our program"? What does that phrase actually mean? What is the bounds of "our program" anyway? Couldn't it be the set {main.c, winkle.h}
A C program is not required to be translated in its entirety at the same time. The text of the program
is kept in units called source files, (or preprocessing files) in this document. A source file together
with all the headers and source files included via the preprocessing directive #include is known
as a preprocessing translation unit. After preprocessing, a preprocessing translation unit is called a
translation unit. Previously translated translation units may be preserved individually or in libraries.
The separate translation units of a program communicate by (for example) calls to functions whose
identifiers have external linkage, manipulation of objects whose identifiers have external linkage, or
manipulation of data files. Translation units may be separately translated and then later linked to
produce an executable program.
> Couldn't it be the set {main.c, winkle.h}
No; in this discussion it is important that <winkle.h> is understood not to be part of the program; no such header is among the files presented for translation, linking and execution. Thus, if the implementation doesn't resolve #include <winkle.h> we get the uninteresting situation that a constraint is violated.
Let's focus on the situation where it so happens that #include <winkle.h> does resolve to something in the implementation.
The bit of the standard that you've quoted says that the program consists of all files that are compiled into it, including all files that are found by the #include directive. So, if <winkle.h> does successfully resolve to something, then it must be part of the program by definition because that's what "the program" means.
Your question about an include file that isn't part of the program just doesn't make any sense.
(Technically it says that those files together make up the "program text". As my other comment says, "program" is the binary output.)
Do you just meant an attempt to include a file path that couldn't be found? That's not a correct usage of the term "program" – that refers to the binary output of the compilation process, whereas you're taking about the source files that are the input to the compilation. That sounds a bit pedantic but I really didn't understand what you meant.
I just checked, and if you attempt to include a file that cannot be found (in the include path, though it doesn't use that exact term) then that's a constraint violation and the compiler is required to stop compilation and issue a diagnostic. Not undefined behaviour.
I don't get your point then. If the file is found then there is no undefined behaviour in the process of the file being included. There might be undefined behaviour in the overall translation unit after the text has been substituted in, but that's nothing to do with the preprocessor.
A couple of solutions in development (but already usable) that more effectively address UB:
i) "Fil-C is a fanatically compatible memory-safe implementation of C and C++. Lots of software compiles and runs with Fil-C with zero or minimal changes. All memory safety errors are caught as Fil-C panics."
"Fil-C only works on Linux/X86_64."
ii) "scpptool is a command line tool to help enforce a memory and data race safe subset of C++. It's designed to work with the SaferCPlusPlus library. It analyzes the specified C++ file(s) and reports places in the code that it cannot verify to be safe. By design, the tool and the library should be able to fully ensure "lifetime", bounds and data race safety."
"This tool also has some ability to convert C source files to the memory safe subset of C++ it enforces"
Fil-C is interesting because as you'd expect it takes a significant performance penalty to deliver this property, if it's broadly adopted that would suggest that - at least in this regard - C programmers genuinely do prioritise their simpler language over mundane ideas like platform support or performance.
The resulting language doesn't make sense for commercial purposes but there's no reason it couldn't be popular with hobbyists.
Well, you could also treat Fil-C as a sanitiser, like memory-san or ub-san:
Run your test suite and some other workloads under Fil-C for a while, fix any problems report, and if it doesn't report any problems after a while, compile the whole thing with GCC afterwards for your release version.
Right. And of course there are still less-performance-sensitive C/C++ applications (curl, postfix, git, etc.) that could have memory-safe release versions.
But the point is also to dispel the conventional wisdom that C/C++ is necessarily intrinsically unsafe. It's a tradeoff between safety, performance and flexibility/compatibility. And you don't necessarily need to jump to a completely different language to get a different tradeoff.
Fil-C sacrifices some performance for safety and compatibility. The traditional compilers sacrifice some safety for performance and flexibility/compatibility. And scpptool aims to provide the option of sacrificing some flexibility for safety and performance. (Along with the other two tradeoffs available in the same program). The claim is that C++ turns out to be expressive enough to accommodate the various tradeoffs. (Though I'm not saying it's always gonna be pretty :)
Even with UB holes plugged, C (and C++) are still unsafe, because there are many assumptions you might want to make that you can not encode in the language.
To get an example that's easy to understand: before the introduction of the 'const' keyword, you just couldn't express that some variable should never be changed. And no amount of UB sanitisers would have fixed this for you: you just couldn't express the concept. There's lots of other areas of these languages that are still in a similar state.
Eg there's no way to express that a function should be pure, ie not have side effects (but is allowed to use mutation internally).
Yeah, but C++ now supports "user-defined" annotations which effectively allow you to add the equivalent of any keyword you need, right? (Even if it's not the prettiest syntax.) For example, the scpptool static analyzer supports (and enforces) lifetime annotations with similar meaning to Rust's lifetime annotations.
I believe gcc actually does support `__attribute__ ((pure))` to indicate function purity. (I assume it doesn't actually enforce it, but presumably it theoretically could at some point.)
They at least fixed this in c++26.
No longer UB, but "erroneous behavior".
Still some random garbage value (so an uninitialized pointer will likely lead to disastrous results still), but the compiler isn't allowed to fuck up your code, it has to generate code as if it had some value.
It won't be a "random garbage value" but is instead a value the compiler chose.
In effect if you don't opt out your value will always be initialized but not to a useful value you chose. You can think of this as similar to the (current, defanged and deprecated as well as unsafe) Rust std::mem::uninitialized()
There were earlier attempts to make this value zero, or rather, as many 0x00 bytes as needed, because on most platforms that's markedly cheaper to do, but unfortunately some C++ would actually have worse bugs if the "forgot to initialize" case was reliably zero instead.
The classic thing is, we're granting user credentials - maybe we're a login proces, or a remote execution helper - and we're on Unix. In some corner case we forget to fill out the user ID. So it's "random noise". Maybe in the executable distributed to your users it was 0x4C6F6769 because the word "Login" was in that memory in some other code and we never initialized it so...
Bad guys find the corner case and they can now authenticate as user 0x4C6F6769 which doesn't exist and so that's useless. But - when we upgrade to C++ 26 with the hypothetical zero "fix" now they're root instead!
Rust here rust there. We are just talking about C not rust. Why we have to using rust. If you talking memory safety why there is no one recommends Ada language instead of rust.
Even within the rust OSS community it's irritating. They will try to cancel people for writing libs using `unsafe`, and makes APIs difficult to use by wrapping things in multiple layers of traits, then claim using other patters are unsafe/unsound/UB. They make claims that things like DMA are "advanced topics", and "We haven't figured it out yet/found a good solution yet". Love rust/hate the Satefy Inquisition. Or say things like "Why use rust if you don't use all the safety-features and traits"... which belittles rust as a one-trick lang!
A small nit: the development of Unix began on the PDP-7 in assembly, not the PDP-11.
(The B language was implemented for the PDP-7 before the PDP-11, which are rather different machines. It’s sometimes suggested that the increment and decrement operators in C, which were inherited from B, are due to the instruction set architecture of the PDP-11, but this could not have been the case. Per Dennis Ritchie:¹
> Thompson went a step further by inventing the ++ and -- operators, which increment or decrement; their prefix or postfix position determines whether the alteration occurs before or after noting the value of the operand. They were not in the earliest versions of B, but appeared along the way. People often guess that they were created to use the auto-increment and auto-decrement address modes provided by the DEC PDP-11 on which C and Unix first became popular. This is historically impossible, since there was no PDP-11 when B was developed. The PDP-7, however, did have a few “auto-increment” memory cells, with the property that an indirect memory reference through them incremented the cell. This feature probably suggested such operators to Thompson; the generalization to make them both prefix and postfix was his own.
Another person puts it this way:²
> It's a myth to suggest C’s design is based on the PDP-11. People often quote, for example, the increment and decrement operators because they have an analogue in the PDP-11 instruction set. This is, however, a coincidence. Those operators were invented before the language [i.e. B] was ported to the PDP-11.
In any case, the PDP-11 usually gets all the love, but I want to make sure the other PDPs get some too!)
We switched to Rust.
Generally, are there specific domains or applications where C/C++ remain preferable? Many exist—but are there tasks Rust fundamentally cannot handle or is a weak choice?
Yes, all the industries where C and C++ are the industry standards like Khronos APIs, POSIX, CUDA, DirectX, Metal, console devkits, LLVM and GCC implementation,....
Not only you are faced with creating your own wrappers, if no one else has done it already.
The tooling, for IDEs and graphical debuggers, assumes either C or C++, so it won't be there for Rust.
Ideally the day will come where those ecosystems might also embrace Rust, but that is still decades away maybe.
Advantages of C are short compilation time, portability, long-term stability, widely available expertise and training materials, less complexity.
IMHO you can today deal with UB just fine in C if you want to by following best practices, and the reasons given when those are not followed would also rule out use of most other safer languages.
This is a pet peeve, so forgive me: C is not portable in practice. Almost every C program and library that does anything interesting has to be manually ported to every platform.
C is portable in the least interesting way, namely that compilers exist for all architectures. But that's where it stops.
> C is not portable in practice. Almost every C program and library that does anything interesting has to be manually ported to every platform.
I'm guessing you mean that every cross-platform C codebase ends up being plastered in cascading preprocessor code to deal with OS and architecture differences. Sure that's true, you still have to do some porting work regardless of the language you chose.
But honestly, is there any language more portable than C? I struggle to come up with one.
If someone told me "I need a performant language that targets all major architectures and operating systems, but also maybe I want to run it on DOS, S390X, an old Amiga I have in my closet, and any mystery-meat microcontroller I can find." then really wouldn't have a better answer for them than C89.
If "portability" to you has to include incredibly esoteric architectures in 2025, then what C has to offer is probably the best you can do, but my point is it doesn't do any better on mainstream platforms either.
If you are targeting any recent platform, both Rust and Zig do what you want.
Compilers existing is essential and not trivial (and also usually then what other languages build on). The conformance model of C also allows you to write programs that are portable without change to different platforms. This is possible, my software runs on 20 different architectures without change. That one can then also adopt it to make use of specific features of different platforms is quite natural in my opinion.
It is essential and nontrivial, but it's also the extremely bare minimum.
You cannot write portable code without platform-specific and even environment-specific adaptations, like handling the presence of certain headers (looking at you, stdint.h and stddef.h), and let's not even start about interacting with the OS in any way.
> IMHO you can today deal with UB just fine in C if you want to by following best practices
In the other words, short compilation time has been traded off with wetware brainwashing... well, adjustment time, which makes the supposed advantage much less desirable. It is still an advantage, I reckon though.
Never meant to be hostile (if I indeed were, I would have question every single word), but sorry for that.
I mean to say that best practices do help much but learning those best practices take much time as well. So short compilation time is easily offseted by learning time, and C was not even designed to optimize compilation time anyway (C headers can take a lot to parse and discard even when unused!). Your other points do make much more sense and it's unfortunate that first points are destructively interfering each other, hence my comment.
Sorry, maybe I misread your comment. There are certainly languages easier to learn than C, but I would not say C++ or Rust fall into this category. At the same time, I find C compilation extremely fast exactly because of headers. In C you can split interface and implementation cleanly between header and c-file and this enables efficient incremental builds. In C++ most of the implementation is in headers, and all the template processing is order of magnitude more expensive than parsing C headers. Rust also does not seem to have proper separate compilation.
Rust encourages a rather different "high-level" programming style that doesn't suit the domains where C excels. Pattern matching, traits, annotations, generics and functional idioms make the language verbose and semantically-complex. When you follow their best practices, the code ends up more complex than it really needs to be.
C is a different kind of animal that encourages terseness and economy of expression. When you know what you are doing with C pointers, the compiler just doesn't get in the way.
> Pattern matching should make the language less verbose, not more.
In the most basic cases, yes. It can be used as a more polished switch statement.
It's the whole paradigm of "define an ad-hoc Enum here and there", encoding rigid semantic assumptions about a function's behaviour with ADTs, and pattern matching for control-flow. This feels like a very academic approach and modifying such code to alter its opinionated assumptions isn't funny.
Yes, based on a few attempts chronicled in articles from different sources, Rust is a weak choice for game development, because it's too time-consuming to refactor.
Basically all of those problems originate with the tradition of conflating pointers and object identity, which is a problem in Rust as soon as you have ambiguous ownership or incongruent access patterns.
It's also very often not the best way to identify objects, for many reasons, including performance (spatial locality is a big deal).
These problems go away almost completely by simply using `EntityID` and going through `&mut World` for modifications, rather than passing around `EntityPtr`. This pattern gives you a lot of interesting things for free.
The video I linked to is long but goes through all of this.
Pretty much nobody writing games in C++ uses raw pointers in entities to hold references to other related entities, because entities can be destroyed at any time and there's no simple way for a referring entity to know when a referenced entity is destroyed.
Using some sort of entity ID or entity handle is very common in C++, the problem is that when implementing this sort of system in Rust, developers often end up having to effectively "work around" the borrow checker, and they end up not really gaining anything in terms of correctness over C++, ultimately defeating the purpose of using Rust in the first place, at least for that particular system.
Can you give an example of what problems need workarounds here?
The benefits seem pretty massive, at least on the surface. For example, you can run any system that only takes `&World` (i.e., immutable access) in parallel without breaking a sweat.
"Mit dem Angriff Steiner's wird das alles in Ordnung kommen" ;)
As shitty as C++ is from today's PoV, the entire gaming industry switched over within around 3 years towards the end of the 90s. 6..7 years is a long time, and a single engine (especially when it's more or less just a runtime without editor and robust asset pipeline) won't change the bigger picture that Rust is a pretty poor choice for gamedev.
I was there Gandalf... ;) Console SDKs offering C or C++ APIs doesn't really matter, because you can call C APIs from C++ just fine. So the language choice was a team and engine developer decision, not a platform owner decision (as it should be).
From what I've seen, around the late mid-90's, C++ usage was still rare, right before 2000 it was already common and most middleware didn't even offer C APIs anymore.
Of course a couple of years later Unity arrived and made the gamedev language choice more complicated again.
The articles describe how the problem is inherent in the language.
If we exclude AAA games, probably the vast majority of the games nowadays don't need manual memory management for the game core (C# was a popular choice, it seems). I guess that if one really needs manual memory management, languages with moderate memory safety would be a more appropriate choice (support libraries/frameworks being equal, which certainly aren't).
I've used Bevy, and ECS is not an appopriate choice for every game (I wouldn't actually advise it unless there is a specific need). It requires very careful design over the whole lifecycle (ECS-based games very easily tend to get a mess), which is exactly the opposite of one wants for rapid prototyping.
There are lots of C and particularly C++ ways, but you're still restricted. Want to use methods in C: nope, you can't. Want language-level tagged unions and pattern matching in either language: nope. Same for guaranteed tail call optimisation and a bunch of other things.
This is especially true for C which supports almost nothing (it doesn't even have a sensible array type!). But is also true for C++: while it supports a lot, it doesn't support everything.
The funny part is that all of these things are easy to achieve as libraries/paradigms.
Methods in C, just have function pointers as members. Common in many codebases.
Guaranteed tail calls, all the compilers guarantee that function calls that are a return expression are tail calls.
Tagged union in C++, it's trivial as a library, see std::variant for a bad example of it, and all the various monadic/pattern-matching variants (pun intended) people have written. C is at a disadvantage here due to lack of lambdas, but I'm sure people have built stuff using some GCC extensions.
what changes, in your opinion, would need to be made to the C array type to make it "sensible"? C's array is simplistic, but I don't think it's not "sensible"...
If you wanted to develop a cross-platform native desktop / mobile app in one framework without bundling / using a web browser, only QT comes to mind, which is C++. I think there are some bindings though.
An application domain where C++ is notably better is when the ownership and lifetimes of objects are not knowable at compile-time, only being resolvable at runtime. High-performance database kernels are a canonical example of code where this tends to be common.
Beyond that, recent C++ versions have much more expressive metaprogramming capability. The ability to do extensive codegen and code verification within C++ at compile-time reduces lines of code and increases safety in a significant way.
I find with C/++ I have to compile to find warnings and errors, while with Rust I get more information automatically due to the modern type and linking systems. As a result I compile Rust significantly less times which is a massive speed increase.
Rusts tooling is hands down better than C/++ which aids to a more streamlined and efficient development experience
Not the GP, but the biggest one is dependency management. Cargo is just extremely good.
As for the language tooling itself, static and runtime analyzers in C and C++ (and these are table stakes at this point) do not come close to the level of accuracy of the Rust compiler. If you care about writing unsafe code, Miri is orders of magnitude better at detecting UB than any runtime analyzer I've seen for C and C++.
I am yet to use async in c++, but I did work on a multi threaded c++ project for a few years
Rust is nicer for async and MT than c++ in every way. I am pretty sure.
But it's still mid. If you use Rust async aggressively you will struggle with the borrow checker and the architecture results of channel hell.
If you follow the "one control thread that does everything and never blocks" you can get far, but the language does not give you much help in doing that style neatly.
I have never used Go. I love a lot of Go projects like Forgejo and SyncThing. Maybe Go solved async. Rust did not. C++ did not even add good tagged unions yet.
Go (at least before generics) was really annoying to use.
Doing anything concurrent in Go is also really annoying (be that async or with threads), because everything is mutable. Not just by default but always. So anything shared is very dangerous.
Async rust is amazing as long as you only mix in one more hard concept. Be it traits, generics or whatever. You can confidently write and refactor heavily multithreaded code without being deathly afraid of race conditions etc. and it is extremely empowering.
The problem comes when trying to write async generic traits in a multithreaded environment.
Then just throwing stuff at the wall and hoping something sticks will quickly lead you into despair.
embedded hardware, any processor Rust doesn't support (there are many), and any place where code size is critical. Rust has a BIG base size for an application, uselessly so at this time. I'd also love to see if it offered anything that could be any use in those spaces - especially where no memory allocation takes place at all. C (and to a lesser extent C++) are both very good in those spaces.
You can absolutely make small rust programs, you just have to actually configure things the right way. Additionally, the Rust language doesn’t have allocation at all, it’s purely a library concern. If you don’t want heap allocations, then don’t include them. It works well.
The smallest binary rustc has produced is like ~145 bytes.
That is far from my only concern. But it's good to see Rust is finally paying attention to binary sizes. And the overwhelming complexity of rust code is definitely not a gain when one is working in embedded spaces anyway. I am however really REALLY annoyed with the aggressive sales tactics of the rust community.
They are pretty much the best choice for prototyping 3D apps and GPU algorithms. They're fast, powerful, and don't impose restrictions - you can do whatever and however. It also helps that CUDA is C++.
I don’t buy the “it’s because of optimization argument”.
And I especially don’t buy that UB is there for register allocation.
First of all, that argument only explains UB of OOB memory accesses at best.
Second, you could define the meaning of OOB by just saying “pointers are integers” and then further state that nonescaping locals don’t get addresses. Many ways you could specify that, if you cared badly enough. My favorite way to do it involves saying that pointers to locals are lazy thunks that create addresses on demand.
No, it's absolutely because of optimization. For instance, C++20 defined signed integer representation as having two's complement, but signed integer overflow is still undefined behaviour. The reason is that if you compile with flags that make it defined, you lose a few percentage points of performance (primarily from preventing loop unrolling and auto-vectorization).
Same thing with e.g. strict aliasing or the various UB that exists in the standard library. For instance, it's UB to pass a null pointer to strlen. Of course, you can make that perfectly defined by adding an `if` to strlen that just returns 0. But then you're adding a branch to every strlen, and C is simply not willing to do that for performance reasons, so they say "this is UB" instead.
Pretty much instance of UB in standard C or C++ is because making it defined would either hamper the optimizer, or it would make standard library functions slower. They don't just make things UB for fun.
This isn’t the reason why the UB is in the spec in the first place. The spec left stuff undefined to begin with because of lack of consensus over what it should do.
For example the reason why 2s complement took so long is because of some machine that ran C that still existed that was 1s complement.
> The reason is that if you compile with flags that make it defined, you lose a few percentage points of performance (primarily from preventing loop unrolling and auto-vectorization).
I certainly don’t lose any perf on any workload of mine if I set -fwrapv
If your claim is that implementers use optimization as the excuse for wanting UB, then I can agree with that.
I don’t agree that it’s a valid argument though. The performance wins from UB are unconvincing, except maybe on BS benchmarks that C compilers overtune for marketing reasons.
> For example the reason why 2s complement took so long is because of some machine that ran C that still existed that was 1s complement.
You're misunderstanding me: as of C++20, there is no other representation in C++ for signed integers other than two's complement (no signed ones' complement, no signed magnitude, nothing else), but signed overflow is still UB. It's not because of obscure machines or hardware, such hardware is not relevant for C++20 and later. The reason for it is performance. From the accepted paper [1]:
> The following polls were taken, and corresponding modifications made to the paper. The main change between [P0907r0] and the subsequent revision is to maintain undefined behavior when signed integer overflow occurs, instead of defining wrapping behavior. This direction was motivated by:
> * Performance concerns, whereby defining the behavior prevents optimizers from assuming that overflow never occurs
You may disagree, you may think they're wrong, but their motivation is performance, that's why this is UB. It's right there in black and white. This was C++, not C, but it's not at all unthinkable that the C standard will also mandate two's complement at some point, and if they do, they almost certainly keep signed overflow undefined for exactly the same reason.
It's not hard to write code that optimizes much better when you use signed loop variables. One of my favorite examples is this function [2] to turn a 3D mesh inside out by flipping the edges of each triangle in a triangle mesh. The godbolt link has two versions of the same function, one with a signed loop variable, one with an unsigned one. The signed one auto-vectorizes and optimizes much better because it can assume that the loop variable never overflows (this version is C++, it's trivial to rewrite it in C and get the same results).
I agree that the stated motivation for continuing to keep UB is performance.
I know that this is misguided based on my own perf tests and others’ perf tests.
Also, it’s wrong to say flat out that UB on signed ints is somehow necessary for perf when even a simple perf test shows that it just doesn’t matter, and the optimization it enables is quite obscure.
I wish there was a way to opt into undefined behavior for unsigned overflow. Its rare that wraparound is actually what you want and in many cases overflow is still a bug. Sucks to have to either miss out on potential optimizations or miss out on the guarantee that the value can't be negative.
You do need some way to overflow properly, because sometimes that is what you want. A common example would be PRNGs, which frequently rely on overflow (the classic LCG, for instance). You could argue that should just be a library function or something (e.g. `add_with_overflow`), though that's more C++ than C.
You are absolutely, 100% correct though: I've never seen a case where accidental overflow doesn't start causing bugs anyway. Like, the Pac-Man kill screen is caused by a byte overflowing (it happens on level 256), and the game goes insane. Pac-Man was written in assembly where overflow is defined behavior, but that doesn't matter at all, the game is still broken. If signed overflow is essentially always a bug anyway, why not make it UB and optimize around it? Especially since it is super-valuable in being able to unroll loops.
People always bring up signed integer overflow as an argument for why UB is scary, and it always seemed like such a bad argument to me. Like, I can understand why people think UB has gone too far in C/C++, but signed overflow is such a bad example. It's one of the most sensible bits of UB in the entire standard, IMHO.
Unlike C++ and (until Martin's work is moved to the actual language ISO document rather than separate) C the Rust language actually has a definition for how provenance is supposed to work.
The definition isn't deemed complete because of aliasing. AIUI The definition we have is adequate if you're OK with treating all edge cases for "Is this an alias?" as "Yes" but eventually Rust will also need to carefully nail down all those edge cases so that you can tread closer without falling off.
I didn't claim that. What I mean is that if a pointer escapes into an inlined function and no further, it will still prevent further optimizations if we apply your rule that only non-escaping locals don't get addresses. The main benefit of inlining is that it is effectively a simple way to do interprocedurally optimizations. I.e.
inline void add(int* to, int what) { *to += what; }
void foo();
void bar() {
int x = 0;
add(&x, 1);
foo();
return x;
}
By your rules, optimizing bar to return the constant 1 would not be allowed.
I think you’re applying a very strange strawman definition to “nonescaping”. It’s certainly not the definition I would pick.
The right definition is probably something like:
- pointers that come out of the outside world (syscalls) are escaped. They are just integers.
- pointers to locals have provenance. They point to an abstract location. It is up to the implementation to decide when the location gets an integer value (is in an actual address) and what that value is. The implementation must do this no later than when the pointer to the local escapes.
- pointer values passed to the outside world (syscalls) escape.
- pointer values stored in escaped memory also escape, transitively
That’s one possible definition that turns the UB into implementation defined behavior. I’m sure there are others
I think you have a non-standard definition. An escaping pointer is an address that the compiler cannot fully track (directly or indirectly). It could be to a syscall, it could be a separately compiled function (without LTO), it could even be to a function in the same translation unit if the compiler cannot inline that function nor do sufficient intraprocedural analysis.
Again, I'm not a compiler writer, but my understanding is that non escaping variables can be optimized in SSA form, escaped variables are otherwise treated as memory and the compiler must be significantly more conservative.
In any case, whether a pointer escapes or not depends purely on the compiler capabilities and optimization level, so it would not be sane making a code well defined or UB depending on the compiler or optimization level.
edit: to be more concrete, do you think that in my example the constant folding of the return into return 1 should be allowed? And if so, which variant of this code would prevent the optimization and why?
> Second, you could define the meaning of OOB by just saying “pointers are integers"
This means losing a lot of optimisations, so in fact when you say you "don't buy" this argument you only mean that you don't care about optimisation. Which is fine, but this does mean the "improved" C isn't very useful in a lot of applications, might as well choose Java.
This asserts that UB was deliberately created for optimisation purposes; not to handle implementation differences. It doesn't provide any evidence though and that seems unlikely to me.
The spec even says:
> behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements
No motivation is given that I could find, so the actual difference between undefined and implementation defined behaviour seems to be based on whether the behaviour needs to be documented.
I'd say the original intent of UB was not the sort of "optimizer exploits" we see today, but to allow wiggle room for supporting vastly different CPUs without having to compromise runtime performance or increasing compiler complexity to balance performance versus correctness. Basically an escape hatch for compilers. The difference to IB also has always been quite fuzzy.
Also the C spec has always been a pragmatic afterthought, created and maintained to establish at least a minimal common feature set expected of C compilers.
The really interesting stuff still only exists outside the spec in vendor language extensions.
I, once again, disagree with the premise that UB is a necessary precondition for optimisation, or that it exists to allow for optimisation. You do not need UB to unroll a loop, inline a function, lift an object or computation out of a loop, etc. Moreover, _most_ UB does not assist in optimisation at all.
The two instances where UB allows for optimisation are as follows:
1. The 'signed overflow' UB allows for faster array indexing. By ignoring potential overflow, the compiler can generate code that doesn't check for accidental overflow (which would require masking the array index, recomputing the address on each loop iteration). I believe the better solution here would be to introduce a specific type for iterating over arrays that will never overflow; size_t would do fine, and making signed overflow at least implementation defined, if not outright fully defined, after a suitable period during which compilers warn if you use a too-small type for array indexing.
2. The 'aliasing' UB does away with the need to read/write values to/from memory each time they're used, and is extremely important to performance optimisation.
But the rest? Most of it does precisely nothing for performance. At 'best', the compiler uses detected UB to silently eliminate code branches, but that's something to be feared, not celebrated. It isn't an optimisation if it removes vital program logic, because the compiler could 'demonstrate' that it could not possibly take the removed branch, on account of it containing UB.
The claim in the linked article ("what every C programmer should know") that use of uninitialized variables allows for additional optimisation is incorrect. What it does instead is this: if the compiler see you declare a variable, and then reading from it before writing to it, it has detected UB, and since the rule is that "the compiler is allowed to assume UB does not occur", use that as 'evidence' that that code branch will never occur and can be eliminated. It does not make things go faster; it makes them go _wrong_.
Undefined behaviour, ultimately, exists for many reasons: because the standards committee forgot a case, because the underlying platforms differ too wildly, because you cannot predict in advance what the result of a bug may be, to grandfather in broken old compilers, etc. It does not, in any way, shape, or form, exist _in order to_ enable optimisation. It _allows_ it in some cases, but that is, and never was, not the goal.
Moreover, the phrasing of "the compiler is allowed to assume that UB does not occur" was originally only meant to indicate that the compiler was allowed to emit code as if all was well, without introducing additional tests (for example, to see if overflow occurred or if a pointer was valid) - clearly that would be very expensive or downright infeasible. Unfortunately, over time this has enabled a toxic attitude to grow that turns minor bugs into major disasters, all in the name of 'performance'.
The two bullet points towards the end of the article are both true: the compiler SHOULD NOT behave like an adversary, and the compiler DOES NEED license to optimize. The mistake is thinking that UB is a necessary component of such license. If that were true, a language with more UB would automatically be faster than one with less. In reality, C++ and Rust are roughly identical in performance.
C and C++ are languages that brought us UNIX, the Linux kernel, macOS and Windows, the interpreters of virtually every other language in the world, powering virtually all software in the world as well as the vast majority of embedded devices.