> C23 furthermore gives the compiler license to use an unreachable annotation on one code path to justify removing, without notice or warning, an entirely different code path that is not marked unreachable: see the discussion of puts() in Example 1 on page 316 of N3054.9
I don't agree with that description at all. Here's the code:
1 if (argc <= 2)
2 unreachable();
3 else
4 return printf("%s: we see %s", argv[0], argv[1]);
5 return puts("this should never be reached");
The only code path that's "entirely different" is lines 1,4,5 and in that case of course you remove a return that's after a return.
And the other valid code path is 1,2,5, which has `puts` after `unreachable`.
To need `puts` you have to imagine a code path that gets past the "if" without taking either branch?
Maybe the author means something by "code path" that's very different from how I interpret it?
I would be pretty surprised if the above code means something different from:
if (argc <= 2) {
unreachable();
return puts("this should never be reached");
} else {
return printf("%s: we see %s", argv[0], argv[1]);
return puts("this should never be reached");
}
This reminds me of a point made by the late Stan Kelly-Bootle, who for years wrote the Devil's Advocate column in UNIX Review magazine. In the early 1990s, he was discussing Microsoft's new C compiler and noted that in the promo material for the new compiler, it showed a benchmark for a loop that counted from 1 to 10,000 then printed "Hello". MS claimed that without optimization it took a few milliseconds, after optimization: 0 ms. A small asterisk explained the optimizer simply removed the loop. Kelly-Bootle pointed out, that the only reason a developer would write such a loop was to introduce a needed delay. Therefore, deleting the loop was not optimizing, but in fact pessimizing. And so, it was in fact Microsoft's Pessimizing C compiler.
Of course, that's technically incorrect. The way the standards are written, the compiler is free to replace the program with any other program that has the same (in a precisely defined sense) observable behavior (these are the famous "as if" formulations in language specs). Heating up the CPU is not considered observable behavior.
If someone really just wants a delay, it's easy to either (for programs running on normal OSs) call a sleep function, or (on tiny embedded systems) add an empty inline assembler statement that the compiler can't see through.
Of course these things are “observable” in the literal sense. And yet, they aren’t considered to be observable by the memory model of any language spec that I know of. Same as CPU power draw, which has been used as a side-channel to extract bits of crypto keys, and is very much influenced by common optimizations.
Practically, if you need to execute a specific sequence of machine instructions in order to prevent side-channel attacks, then you have to rely on assembler, compiler intrinsics and/or OS support. But that was true way before Spectre.
Those delay loops are common on microcontrollers and the usual solution is to either make the counter volatile or insert something opaque to the compiler in the loop body.
It would be of course nice if a warning was produced for that specific case: This whole loop was removed - is it really what you wanted, or is it a broken delay loop?
I think it's a practical example of how the C language has made a journey to being more high abstraction than it used to be, in practice. And how that unsettles those used to the old behaviour.
I think the point is that if the `argc <= 2` path is unreachable, then that means argc is always greater than 2, permitting the compiler to optimize the entire block to just:
return printf("%s: we see %s", argv[0], argv[1]);
IOW, the conditional has been elided. But you're right in that the wording of the complaint doesn't match the example. The author presumably had in mind some of the more infamous NULL pointer-related optimizations, without spending the time to put together a properly analogous example.
in which not just lines 4-6,8 go away (as you said) but also lines 1-2.
It makes sense to me but I can see why the author would characterize this situation as "license to use an unreachable annotation on one code path to justify removing an entirely different code path that is not marked unreachable". In a different world one might expect A to be printed "before the UB happens".
On the other hand, that has been the behaviour of optimising compilers in the face of UBs for years at this point, decades maybe. The linux kernel was hit by a deref' constraint propagation back in 2009 or so.
This is a behaviour I would absolutely expect from the construct, I would even qualify it as "the point".
I find this especially surprising because line 2 may be exactly the reason why line 5 is unreachable. E.g. if puts("A") contractually throws an exception you cannot just remove it.
One way to look at it (and I am not sure if this is correct, but it may be what the essay author meant) is to not treat the `unreachable` as affecting the presence of the decision, but only the result of the decision. If `unreachable` was replaced by a normal statement, we'd have:
if (argc <= 2)
do_something();
else
return printf("%s: we see %s", argv[0], argv[1]);
So the `return printf` is executed when `argc` is greater than 2. If we remove just the body of the first branch:
if (argc <= 2)
;
else
return printf("%s: we see %s", argv[0], argv[1]);
the same thing holds. And additionally when `argc <= 2`, control will move past the `if`.
Under this view, if the `unreachable` won't cause the entire removal of the `if`, the compiler will produce the equivalent of:
if (argc > 2)
return printf("%s: we see %s", argv[0], argv[1]);
return puts("this should never be reached")
Again, I don't say this is the correct interpretation, but it is one possibility, that would have to be ruled out by other parts of the standard.
I understand that interpretation, but that's what the end of my comment is about. If we treat unreachable as affecting the block it's in, but pretend it's not there for control flow, then the two versions of the code do different things. That's confusing and hard to preserve.
This just shows that "unreachable" is almost impossible to use safely. The only safe use of unreachable is if it is immediately after an instruction that makes the program stop running. It is not for "this cannot happen", because things that "cannot happen" happen all the time. If you use "unreachable", you're just asking for trouble and it seems the compiler authors are happy to oblige.
This couldn't be more wrong. What you say to never use unreachable for is one of the most important use cases of unreachable. The whole point is to give the optimizer an assumption that it can't figure out on its own.
One example of it being useful is unchecked std::variant access in c++ - there isn’t any api to access it like a union (if you already know the type) but you can mark the wrong type path unreachable to the same effect.
There's no problem with this feature. I don't understand TFA's problem with it. As a programmer I get to not use `unreachable()` if I don't want to, and if I do I'm happy that the compiler takes my word for it and does the right thing. This is not at all like code elision in UB cases.
This is not about code that's found to be unreachable through static analysis (where compilers might warn), but about a manual programmer annotation that claims the code is dynamically unreachable even though statically it might look otherwise.
Unreachable is mainly used as an optimization hint. For instance if you put an unreachable into the default branch of a continuous and non-exhaustive (from the pov of the compiler) switch-case statement, the compiler will not emit a range check for the jump table lookup.
If the compiler doesn't know that abort exits the program, they have to compile the normal_stuff path under the assumption that the error path might have run before it. This might result in suboptimal code.
Currently, many compilers support annotations such as __attribute__(noreturn) and __builtin_unreachable() to manually indicate that a code path is unreachable. C23 is now standardizing these features (with a slight tweak to the syntax).
To aid with optimisation, it basically lets you ask the compiler to remove branches, and provide constraints to the same.
An implementation might trap in debug code, but given no context would be provided you'd likely avoid this and would instead use your own wrapper macro to output a message of some sort in that case.
But why put in unreachable?
Doesn't make any sense to me.
If a branch is truly not supposed to ever happen, why have a branch at all? Just remove that code from the source entirely- that helps the optimizer even more, because the most optimal code is of course no code at all.
The predominant focus is realloc(pre,0) becoming UB instead of what the author misleadingly describes as useful, consistent behaviour. It is far from that, and that’s the entire reason that it was declared UB in the first place: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2464.pdf. Note that this wasn’t a proposal to change something, it’s a defect report: the original wording was never suitable.
The second part is the misconception about the impact of UB. Making something UB does not dictate that its usage will initiate the rise of zombie velociraptors. It grants the implementation the power to decide the best course of action. That is, after all, what they’ve been doing all this time anyway.
Note that this deviates from implementation-defined behaviour, because an implementation-defined behaviour has to be consistent. Where implementations choose to let realloc(ptr,0) summon the zombie raptors, they are free to do so. Don’t like it? Don’t target their implementation. Again, this isn’t a change from the POV of implementers - it’s a defect in the existing wording.
In this case, the course of action that any implementation will choose is to stick with the status quo. It is clearly not a deciding factor in whether or not you embrace the new standard, and to suggest otherwise is dishonest, sensationalist nonsense. The feature was broken, and it’s just being named as such.
I agree that realloc was poorly defined for the 0 size case, I think UB or IDB both would have worked in this case to really drive that point home, the WG chose UB.
That being said, you're completely wrong about what UB means. Making use of UB may as well initiate the rise of zombie velociraptors. Except for the situation where your implementation explicitly specifies that it provides a predictable behaviour for a specific case of UB, there's literally no guarantee of what will happen. Assuming that the implementation will stick with some status quo and your code won't exhibit absolutely unusual behaviour is just naiive.
Please don't mislead people into thinking that it's ever a good idea to assume that undefined behaviour will be handled sensibly, this kind of mislead assumption is one of the major sources of bugs in C code.
> this kind of mislead assumption is one of the major sources of bugs in C code.
This is not even close to be true. Most bugs in C code are from programmer mistakes, not from UB behavior. The exaggeration that is spread by some people regarding UB is close to absurd. If something is UB, it may generate different results in different situations, even with the same compiler. The standard is just clarifying this problem. A good compiler will do something sensible, or at least issue a warning when this situation is detected. If you have a bad compiler that does strange things with your code, it's not a defect of UB but the compiler instead.
Optimizing compilers don’t work like that. They can either deviate from the standard and leave it as defined behavior, or mark it UB and go with it as usual.
To get some insight by analogy, consider this set of constraints (unrelated to C):
x <= 7
2x >= 5
…(more with x, y, z but not more constraining x)…
When you feed this to a linear constraint solver, you may get anything from 2.5 to 7 as x. E.g. 3.1415926. Not because a solver wanted to draw some circles, but because it transformed your geometric problem into an abstract representation for its own algorithm, performed some fast operations over it and returned the result. Nobody knows how exactly a specific solving method will behave wrt (underconstrained) x given that the description above is all you have.
When you feed UB into an optimizer, you feed a bit of lava into a plastic pipe, figuratively. You’ll get anything from program #2500…0000 to program #6999…9999, where “…” is few more thousands/millions of digits. Run some numbers from there as an .exe to see if something absurd happens.
The nature of UB and optimizers is that you either relax UBs into DBs and get worse efficiency, or you specify more UBs and get worse programming safety. What happens in between can be perceived as completely random. And the better/faster the optimizer is, the more random the outcome will likely be.
The exaggeration that is spread by some people regarding UB is close to absurd
UB-in-code is absurd by definition, no exaggeration here.
> Most bugs in C code are from programmer mistakes
These most often lead to the triggering of UB. The reason why programmer mistakes lead to confusing bugs instead of simple and straightforward bugs which are easy to catch in the development process is mainly because UB imposes no restrictions on what the compiler should do. In the vast majority of UB cases the compilers simply don't do anything, and assume it can't happen. This is why dereferencing a pointer and then checking if it's null ends up eliding the null check (because if you've dereferenced it, it can't be null, that would be UB). Accessing past the end of an array is UB so it can't happen, therefore your compiler won't check for it. Accessing past the end of an array and accidentally reading from/writing to another variable - likewise.
UB encompasses ALL behavior for which the standard does not provide an explicit definition. The reason why the C standard provides explicit instances of UB usually boils down to clarifying situations where people were confused about whether something was UB or not. But if the behaviour is not defined in the standard, then it is by definition UB.
Right, this should have been left to the implementor if they didn't want to standardize one behavior. Making it UB is the worst possible outcome. Yes, people who write portable code will still want to not rely on `realloc()`'s freeing behavior, but if you do and your realloc() implementation doesn't, then you suffer a leak, while if you do and realloc() decides to wipe your drive and make your power supply explode...
> Except for the situation where your implementation explicitly specifies that it provides a predictable behaviour for a specific case of UB, there's literally no guarantee of what will happen.
That situation is "when you have UBSan turned on".
> The second part is the misconception about the impact of UB. [...] It grants the implementation the power to decide the best course of action. That is, after all, what they’ve been doing all this time anyway.
Wrong, Wrong, Wrong.
UB allows the implementation to take any arbitrary course of action, without informing anyone, without documentation, without any conscious decision, without weighing anything to be better/worse. Nondeterministically catching fire and launching nuclear rockets is a completely compliant reaction to UB.
What you are describing is "implementation defined" behavior. That has to be deterministic, documented, and conforming to some definition of sanity. Examples are the binary representation of NULL, sizes of integer types or stuff like the maximum filename length. Sadly, too many things in C have "undefined behavior", too few have "implementation defined" behavior.
And UB has always been an excuse for compilers to screw over programmers in hideous ways. Programmers are rightfully afraid of any kind of new UB being introduced, because it will mean that whole new classes of bugs will arise because the compiler optimized out that realloc(..., a) where a might be 0, because thats UB, so screw you and your code... And this change is especially dangerous because it makes a lot of existing code UB.
And UB has always been an excuse for compilers to screw over programmers in hideous ways
Your reply was great up until this. Compiler writers aren’t looking to screw over programmers, they’re looking to make code faster. UB gives them the ability to make assumptions about what is and is not true, at a particular moment in time, in order to skip doing unnecessary work at runtime.
By assuming that code is always on the happy path, you can cut a lot of corners and skip checks that would otherwise greatly slow down the code. Furthermore, these benefits can cascade into more and more optimizations. Sometimes you can have these large, complicated functions and call graphs get optimized down to a handful of inlined instructions. Sometimes the speedup can be so dramatic that the entire application is unusable without it!
Many of these optimizations would be impossible if compilers were forced to assume the opposite: that UB will occur whenever possible.
The tool programmers have available to them is compiler flags. You can use flags to turn off these assumptions, at the cost of losing out on optimizations, if your code needs it and you’re unable to fix it. But it’s better to turn on all possible warnings and treat warnings as errors, rather than ignoring them, to push yourself to fix the code.
the thing that makes UB almost malicious is that it propagates inter-procedurally. This makes reasoning about code with UB basically impossible which means that you should always assume that the compiler is going to screw you over if you use it because there is no way to know whether it will.
You should consider a program with undefined behaviour to be the equivalent of a mathematical proof that contains an unstated contradiction. Ex falso quodlibet: from a falsehood anything follows. Also called the principle of explosion.
Undefined behaviour renders your entire program meaningless. It must be avoided at all costs. Using undefined behaviour on purpose is like sticking a fork in an electrical socket.
> Undefined behaviour renders your entire program meaningless
That's exactly the complaint. Consider that the implementations of the standard library sometimes have exposed UB: that renders behaviour of all of the running code on the system undefined.
Many programmers believe that the fallout of the UB could, and therefore should, be limited in scope.
It's funny that your original post was an objection to how undefined behavior gives license to screw developers over, but here you are talking about how undefined behavior is like sticking a fork in an electrical socket.
That's not an argument to keep live grenades laying around, it's an argument to remove them from the spec.
Like signed int being UB. Define it to have 2 complement semantics. Problem solved. I'm sure the nutters trying to extend C++ with templates will howl but this is C not C++. And seriously C++ is dead man walking at this point.
I wish UB were only as nasty as "nondeterministic behavior". In fact, if there's UB in anything the compiler sees, nothing at all can be assumed, including whether you even get an output. What you've given the compiler isn't C, so it doesn't have any obligations to do anything with it. The codepath with UB doesn't have to run for the nuclear rockets to launch and the nasal demons to appear.
Since approximately every nontrivial program ever written has UB, in actual practice we're only saved by the fact that compilers aren't entirely maliciously compliant.
That's not true. If the program's execution path from start to finish avoids UB then you're safe. (Also the source code itself has to avoid UB, but that part isn't hard.)
It's true that code with UB does not have to be reached, per se, but it does have to be something your program will reach before it can hurt you.
You're correct in practical terms, but I'm making a very pedantic point about what the standard requires happen, mainly because this pedantry has important implications for e.g. safety critical C. Note 1 to the definition in 3.4.3 provides some clarification about the extent of UB and states that UB can manifest at translation time. It also gives says that the translator should behave in a documented manner when encountering UB, but does not require that it do so.
C has both translation-time UB and runtime UB. (C++ explicitly separates the two concepts into "ill-defined, no diagnostic required" and "undefined behavior".) You can tell them apart from the condition for UB to occur: if it's a translation-time condition, then it's translation-time UB, and if it's a runtime condition, then it's runtime UB. (Same with implicit UB: is it a translation-time or a runtime assumption being violated?)
Usually when we talk about UB, we're implicitly talking about runtime UB, since translation-time UB is generally far less subtle. If a program contains only conditional runtime UB, the compiler is not permitted to break the entire program from the very beginning, since all possible executions that do not trigger runtime UB must execute correctly as per 5.1.2.3.
Fine. HN is, after all, a place where you can be pedantic.
But those of us who are actually writing programs mostly care about "in practical terms", and in practical terms, this doesn't happen, so we don't care. We've got enough trouble worrying about what does happen; we don't have time and energy to worry about what doesn't and won't happen.
> approximately every nontrivial program ever written has UB
You can replace "UB" for "bugs" and the result is the same. UB is a bug on the part of the programmer, from the point of view of C, similar to dereferencing a null pointer. When the standard says that something is UB, it is just clarifying what these situations are.
What the standard explicitly calls out as UB is only a small subset of actual UB.
While you can certainly classify all UB as "bugs", doing so misses the critical differences between UB and other categories of bugs. If you have a logic bug for example, your program will correctly and consistently do the wrong thing. It will continue doing that wrong thing with a different compiler, on a different platform today and 10 years from now. Implementation defined behavior is a bit looser, but will still be consistent with any particular implementation (which will document the behavior) and will only manifest in the code that depends on it. A PR inserting one of these "normal" bugs doesn't invalidate the entire rest of the program.
UB is different. You can't make assumptions about UB because from the point of view of the standard, UB is "not C". There are no assumptions to be made, it's just all the stuff that doesn't have assigned semantics. And since the input is meaningless, so is the entirety of whatever the compiler gives you back.
> If you have a logic bug for example, your program will correctly and consistently do the wrong thing.
Not correct. Bugs can occur differently in different architectures, even in high level languages. UB is just a kind of bug whose effect depends on how the compiler behaves, so you have to be careful to test your code on different compiler settings. This is nothing new on programming languages, it is only made explicit in the C standard. Suddenly people started to believe that pointing out the obvious source of bugs (UB) in the standard is equivalent to let programs misbehave.
Bugs are UB-like in a sense (what's the code going to do? well, you'll have to think about it, or try it and see), but UB is strictly worse than bugs (different compilers, even different versions of the same compiler, can do radically different things way beyond the scope of the bug).
That's exactly why a compiler shouldn't be able to 'optimize' in the face of UB, it should be an ERROR and the section of undefined behavior highlighted in the error message.
This would mean you’d have to insert a check every time you add two signed integers together, because signed overflow is UB. You’d also have to wrap every memory access with bounds checks, because OOB memory access is UB.
There are also tons and tons of loop optimizations compilers do for side-effect free loops which would have to be removed completely. This is because infinite loops without side effects are UB. So if you wanted these optimizations you’d have to prove to the compiler — at compile time — that your loop is guaranteed to terminate since it is not allowed to assume that it will. Without these loop optimizations, numerical C code (such as numpy) would be back in the stone ages of performance.
Edit: I just wanted to point out that one of the new features in C23 is a standard library header called <stdckdint.h> that includes functions for checked integer arithmetic. This allows you to safely write code for adding, subtracting, and multiplying two unknown signed integers and getting an error code which indicates success or failure. This will be the standard preferred way of doing overflow-safe math.
Doing that at compile time would require being able to perfectly predict everything the program can do, which is equivalent to solving the halting problem (make the program do something undefined after it finishes, then if you get an error at compile time then it halts) and is mathematically impossible. Doing it at runtime would have a massive performance impact
Compilers actually have some options to enable that.
The problem is, it only works well in the simplest cases when the code will 100% exhibit UB within a single function.
In most cases, the UB would only manifest on particular input values - if you want your compiler to warn about that then it will report one "potential UB" for every 10 lines of C code, and nobody wants to use such a compiler.
The case of realloc being declared UB (as opposed to impl-defined) was not driven by the compiler writers but by the people who write the C libraries.
This isn't a case of compilers screwing over the programmers, because the people who are responsible for those optimizations are the people who are scratching their heads as to why it's UB and not impl-defined behavior.
int n;
printf("type 0 to stop the rise of zombie velociraptors");
scanf("%d", &n);
realloc(pre, n);
if (n != 0) rise_zombie_velociraptors()
May result in velociraptors raising even if the user enters "0".
The reason is that because realloc(pre, 0) is UB, for the compiler, it cannot happen, so n can't be 0, so the n != 0 test can be optimized out, so, velociraptors.
> The second part is the misconception about the impact of UB. Making something UB does not dictate that its usage will initiate the rise of zombie velociraptors. It grants the implementation the power to decide the best course of action. That is, after all, what they’ve been doing all this time anyway.
Wrong. UB never happens. That is the promise the program writer makes to the compiler. UB never happens. A correct C program never executes UB. This allows the compiler to assume that anything that is UB never happens. Does some branch of your program unconditionally execute realloc(..., 0) after constant propagation? That branch never happens and can just be deleted.
Reading the defect report, they state "Classifying a call to realloc with a size of 0 as undefined behavior would allow POSIX to define the otherwise undefined behavior however they please." which is wrong. UB cannot be defined, if you define it, you are no longer writing standard C. It should instead have been classified as "implementation-defined behaviour".
In any case it's not that hard to just write a sane wrapper. This one is placed in the Public Domain:
void *sane_realloc(void *ptr, size_t sz)
{
if (sz == 0) {
free(ptr); /*free(NULL) is no-op*/
return NULL;
}
if (ptr == NULL) {
return malloc(sz);
}
return realloc(ptr, sz);
}
I am calling it sane and not safe, because it is not safe. You still have the confusion of what happens when the function returns NULL (was it allocation failure or did we free the object?) - check errno. However, it has the same fully defined semantics on most all implementations and acts like people would expect.
You may be tempted to make the function return the value of errno, mark it [[nodiscard]] and take a pointer-to-pointer-to-void, so that the value of the pointer will only be changed if the reallocation was successful. I am not sure if that is safer. You are trading one possible bug - null pointer on allocation failure, which then will cause a segmentation fault for another - stale pointer on allocation failure, but with updated size. The latter is more likely to be used in buffer overflow attacks than the former.
The first sight of "catch fire" might not have caught my attention, but by the time it got to "instrument of arson" and "Molotov cocktails", the style was sufficiently distracting that I was convinced I wasn't the intended audience.
My understanding was that they're changing realloc() because they previously allowed zero length arrays and because you can't tell if this is a zero length array you need to either get rid of zero length arrays or change realloc().
So the feature wasn't broken to begin with, it was broken by another feature.
I actually like unreachable() a lot. What it does is that it invokes undefined behavior, that's all.
It does nothing trickier than any other kind of UB. In fact, I could implement unreachable() like this: void unreachable() { (char *)0 = 1; }.
Standardizing it however gives interesting options for compilers and tool writers. The best use I can find is to bound the values of the argument of a function. For example, if we have "void foo(int a) { if (a <= 0) unreachable(); }, it tells the compiler that a will always be >0 and it will optimize accordingly, but it can also be used in debug builds to trigger a crash, and static analyzers can use that to issue warnings if, for example, we call foo(0). The advantage of using unreachable() instead of any other UB is that the intention is clear.
AFAIK, assert() is not undefined behavior, so it can't be used for optimization. It is either implementation-defined in debug mode, or does nothing in release mode.
For example:
assert(a >= 0);
if (a < 0) printf("a is negative");
In release mode, assert() will be gone, so the if/printf() will stay. If we used "if (a < 0) unreachable();" instead of assert(), it would optimize away both lines.
Using `unreachable()` instead of `assert()` for your preconditions without profiling first is just pre-loading the gun to shoot yourself in the foot in the future. When those preconditions are inevitably violated at some point, you will get random UB corruption rather than simply aborting as is the case for assert.
Yep, undefined behavior is unsafe, C in general is unsafe. There are plenty of languages that are safe, though a little bit rusty like ADA, use one of these if you want safe code.
If you still want to use C, for example for compatibility reasons and want to make it safer, assert isn't going away (unless you set NDEBUG). Preconditions are not "inevitably violated", there are ways of making sure they aren't, and I think an explicit "unreachable()" can help tools that are designed for that purpose.
Should you profile first before using unreachable() for optimization purposes? Maybe, but the important part is that now, you have a way of clearly and effectively tell the compiler what you know will never happen so that it can optimize accordingly, whether it is before or after profiling.
Compilers usually do a great job at optimization, but there are often some edge cases the compiler have to take into account in order to generate code that complies with the C standard, and it can have an impact on performance. unreachable() is one way to tell the compiler "please forget about the edge case, I know it won't happen anyways", the best part is that it is explicit, no obscure tricks here.
Side note about profilers: no matter what your strategy is with regards to optimization, I think profilers are essential tools that don't get enough attention. People talk a lot about linting, coverage and unit tests, but profilers are not to be left out. They are not just tools that tell you where not to optimize your code, they can also find bugs, mostly performance bugs, but not only.
> What it does is that it invokes undefined behavior, that's all. [...] it can also be used in debug builds to trigger a crash
How can it be used to trigger a crash (a specific behavior) if the behavior it invokes is undefined? Are you saying it would be defined differently for debug builds so that it doesn't invoke undefined behavior?
I really don't think anyone could possibly want the _specified behavior_ of a function changing below their feet.
However, the author is unlikely to be correct here. E.g., to this day, glibc contains _multiple implementations of memcpy_ just to satisfy those executables that depend on the older, memmove-like behavior that was once part of the unspecified behavior of glibc. The only way to get the dynamic linker to choose one of the newer versions is to, well, rebuild the executable. It is inconceivable that glibc would not use symbol versioning with an actual specification change.
The behavior is practically the same as with static linking, and you still get the benefits of dynamic linking.
Exactly! Shared libraries mean that new code with modified behavior can and will be called when made available, independent of how the original code was compiled. It is interesting that people come out to complain about this obvious behavior.
The problem isn't changing implementation. This is expected with shared libs. The problem is changing the contract of the function and then expecting it to be drop in compatible. It's not. It _should_ be treated as a breaking ABI change, because the old behavior and new behavior are not compatible, yet it's being masqueraded as such. It's quite literally the same behavior/attitude behind the "w" vs "wt" change that led to aCropolyse.
It's a really weird complaint. The standard specifies that it's now undefined behavior. That imposes zero requirements to change the library. Whatever it is the library was doing, it's one possible undefined behavior.
I’d rather have small binaries and memory efficient systems instead of huge blobs having their own complete disconnected environments with non-coherent behavior on the same situation. Also, wasting tons of memory while at it.
If I have something that critical, I can always statically compile.
> C178 purports to be a bug-fix revision of C11. Does the word "toto" on page 1 indicate (a) the editor's musical tastes; (b) that nobody bothered to spell-check the document; (c) that we're not in Kansas anymore; or (d) none of the above?
As a french guy I'd go with (d).
I've often seen "toto" used as a placeholder name, sometimes followed by "titi", "tata", "tutu", I have even used it myself. It is similar to "foo", "bar", "baz". I don't know if it is specific to France, of French speaking countries, but it is definitely a thing here.
Frankly, the C standards ctte went off the deep end when they effectively banned NULL to memset etc (obv with zero length).
Not because these functions couldn't handle it, but because this assertion simplifies optimizations elsewhere.
This has required adding extra checks in my code, found mainly by trial and error, and has made it less readable and less optimal.
Finally, the checked arithmetic operations returning false on success is a horror show. Fortunately it will be found on the first time the code is run, but that's a damnably low bar :(
The checked operations ask "did an error occur?". If it's false, then the check passed and no error occurred. If it's true, then the check indicated an error.
I find it strange to introduce real bools (which these macros return according to their official signatures) and then to assign them a meaning of a still-nonexistent but widely used C type. At least my C intuition stumbles upon that immediately, no matter how long I think about it.
Ah, anyway, standard C/libc is basically a lost cause. It can’t get any worse, since you have to refer to a manual at every call to not step on a landmine.
Author is angry but not wrong. Lifting the most damning quote from the article as I haven't seen it for a while.
C inventor Dennis Ritchie pointed to several flaws in [ANSI C] ... which he said is a licence for the compiler to undertake agressive opimisations that are completely legal by the committee's rules, but make hash of apparently safe programs; the confused attempt to improve optimisation ... spoils the language.
I'd prefer if it were more letters. It bothers me when API designers omit random letters just to save a few keystrokes. These are particularly egregious because I keep forgetting which letters they kept. Is it "chk"? or "ckd"? or "chd"? or something else?
I wrote a portability library that wraps these with compiler intrinsic and standard C fallbacks. I chose to spell out the full word in addition to making the type explicit. It's a lot more verbose of course but a lot clearer to read:
The other way isn't really definable as an assignment mathematically.
And there is a lot more to it than just pass/fail. First, an addition doesn't fail, from a computer architecture perspective, the addition will always succeed, the only thing that could fail (in all the usual architectures) are possible memory fetch and store operations when not strictly dealing in register or immediate operands. Second, there is no fail flag. There is a overflow flag, an underflow flag, a zero flag, a sign and a few more that are irrelevant here. Any of overflow, underflow, zero or sign might mean that the operation "failed" depending on the types of your operand. Where the processor doesn't know anything about the type, so there won't be a straightforward 'fail' flag in any case. Only the library or compiler can use type information such as (un)signedness, bignum-ness, nonzeroness, desired wraparound (for modular types) and other possible types together with aforementioned flags to decide if that addition might have failed.
So nothing is fundamentally flawed, what you are describing is just insufficiently complex (because there is no fail flag, just a ton of other flags) or overly complex (because uint32_t c = a + b is modular 2^32 arithmetics and cannot fail).
> The other way isn't really definable as an assignment mathematically.
This correction is condescending and unnecessary. Unless the person had never written a single line of code in their life, then they would obviously know "a+b" is not a modifiable lvalue.
And the point about pass/fail was also obviously not mean to capture the full complexity of the flags set by a CPU operation. It was very clearly a statement about how basic addition does not behave in computers the way it does on paper -- as simple as that.
From HN guidelines: "Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize."
You might be right on the first point. Edit: actually, you might not be. There are languages with compound lvalues and CPU architectures with multiple result registers (x86 being the best-known example). E.g. you can do "(result, flags, err) = do_stuff(a, b, c)" in Go, and x86 DIV storing different parts of the division result in different registers: https://c9x.me/x86/html/file_module_x86_id_72.html
And generally with common CPU architectures, flags are another such result register that is always written, such that any operation like c := a+b is actually something like (c, flags) := a+b. And for stuff like multiplication, there is actually the notion of two result registers being the higher and lower part of the resulting operation, like (a * 2^32 + b) = c * d (see x86 MUL). Therefore some precision in language is necessary for the discussion (and yes, the different meanings of ==, =, := in various languages and mathematics are also confusing, even to me ;).
I do strongly disagree on the second one about pass/fail. This kind of nitpicking is necessary here, because the discussion is about a standard intended to precisely describe such operations, and how the underlying hardware might be utilized to execute them. Being imprecise in this context is dangerous, wrong, problematic and leads to the whole point of the discussion being lost in a sea of handwaving.
The type of a would be Integer[5..5], the type of b would be Integer[12..12], the type of c would therefore be Integer[17..17]. In a more complex example:
def foo(a: Integer[0..10], b: Integer[0..10]):
return a + b
The return type of this function would be Integer[0..20].
This kind of type system can solve a number of issues, all but division by zero (which would probably still have to be solved with some kind of optional type).
If type inference dictates that the upper range of an integer would be too large to physically store in a machine data type, then you either resort to bignums or you make it a compilation error. By adding modular and saturating integer types you can handle situations where you want special integer behaviours. By explicitly casting (with the operation returning an optional) you can handle situations where you want to bound the range. This drastically simplifies a lot of code by removing explicit bounds checks in all places except where they are absolutely necessary. If for some reason you care about the space or computational efficiency of the underlying machine type, you can have additional annotations (like C's u?int_(least|fast)[0-9]+_t). If you absolutely must map to a machine type (this is usually misguided, unless you are dealing with existing C interfaces, for which such a language can provide special types) you can have more annotations.
Ada has something resembling this. I believe there are some other languages that implement similar features. I believe this sort of thing has a name, but I am not great with remembering the names of things.
I think the issue with this is that the worst-case bounds normally grow much faster than the actual values. And it can be easy to see for the programmer that the values can't actually grow that much because a is only big when b is small or some property like that, but then you have to convince the compiler of the same. I might be misremembering though.
> because a is only big when b is small or some property like that
Exactly, the expressiveness of the type system then (typically) becomes the obstacle: How do you express that a and b could each reach INT_MAX but their sum never exceeds INT_MAX?
Those kinds of assumptions are where you explicitly cast to a smaller ranged type with the option of an error if the sum does exceed a limit. The point of this type system is not to be able to fully encode every possible interaction between numbers in a system, but rather to remove unnecessary bounds checking in a bunch of cases and make it explicit in the few cases where you ARE actually making an assumption.
> Those kinds of assumptions are where you explicitly cast to a smaller ranged type
But how exactly do you do that? As mentioned, a and b individually can still reach INT_MAX.
I agree with your overall assessment, though. If a type system could represent (and recognize, and evaluate / automatically draw conclusions from) any possible restriction on the values of variables this would probably amount to the type system being able to carry out arbitrary mathematical proofs. The existence of such a type system seems rather unlikely.
But the concept is just a little bit over 30 years old. So don't expect it shows up in most mainstream languages before the end of the next 20 years, and don't expect it to come to the C languages ever.
(Please also note that for this feature both versions don't need language support at all but are "just" libraries, as the language is powerful enough to express all kinds of type level / compile time computations in general.)
I know C is never getting anything like this. But C is really just stuck being a very crappy ABI design language.
I do wish things like Rust had native support for stuff like this though.
And it really doesn't have to get in the way of anyone who insists on the more primitive type systems, but it would be nice to have a high level language which didn't have assembly-level (of abstraction) integer types. Why should I care that my machine works with square multiples of 8 bits at a time or care that they have specific wrapping behaviour.
> But the concept is just a little bit over 30 years old. So don't expect it shows up in most mainstream languages before the end of the next 20 years, and don't expect it to come to the C languages ever.
Any specific results/papers from (refinement) type theory you hope/expect to see implemented in the next 20 years?
"Looking forward, marijuana legalization will surely beget notions such as fractional-, imaginary-, and negative-length objects, each with as much potential for mayhem as zero-length objects."
Maybe I'm being dense. To me it appears that the standards are telling compiler writers what should be done. In doing so the compilers will become ever more complex and thus bug-prone.
I learnt C back when K&R (first edition) was the reference. Ok, it was hardly much more than a universal assembler to make every computer look like a PDP-11. In my experience C is the language to use when you want to be close to the metal. For the rest I use which ever high-level language/environment is best suited. Admittedly some FFI are a pain to use, but once you get the boilerplate bedded down your much higher level language gets the coordination done.
C reached its zenith in C90, and saw a few good ideas in C99. Everything since has been wankery from people who either are bored, or have a severe case of C++-envy.
Even then it was already outdated when compared against languages like Modula-2 and Object Pascal, it got lucky to ride into the waves of UNIX adoption.
> All C standards from C89 onward have permitted compilers to delete code paths containing undefined operations—which compilers merrily do, much to the surprise and outrage of coders.16 C23 introduces a new mechanism for astonishing elision: By marking a code path with the new unreachable annotation,12 the programmer assures the compiler that control will never reach it and thereby explicitly invites the compiler to elide the marked path.
I don't agree with this in the slightest. I'm not "outraged" by undefined behaviour, it's a fundamental tool for writing performant code. Ensuring that dereferencing a null pointer or accessing outside the bounds of an array is undefined behaviour is what lets the compiler not emit a branch on every array access and pointer dereference.
Furthermore, I really don't understand the outrage that there is another explicit tool to achieve behaviour the author may or may not consider harmful. If it's an explicit macro, it's not a tarpit!
While the situation with realloc() is unfortunate, it is also not difficult to write a wrapper that does what the author wants. I’ve done that before, because it has long been known that not all realloc() implementations conform to the (prior) C standard. One can furthermore assume that existing implementations won’t change their behavior just because C23 made it UB.
Is it just me that thinks that the article is a [skilfully drafted] joke (or parody or whatever the correct word is)? The fact that it has been published close to April 1st raises more suspicions.
(don't get me wrong. love C. but in an innocent sort of way, like a teenager quite unaware of betrayals, heartbreak, love triangles, or UB, UsB, and IDB..)
I always wonder how much these new C standards use, as C is now mostly used in areas where one is severely limited when it comes to compiler choice. Where I work, we use GCC 6.2 and iso9899:1990 (C90). If we were able to use a modern compiler, we would probably just use C++.
With this rule, the compiler can conclude that p and q cannot alias, even if it doesn't have body of do_random_stuff. Without it, it would first have to prove that p is never freed before calling q, which is basically impossible (moving the body of intervening code into a different file, for example, would do the trick).
900 years ago there was a CPU which stored pointers in special registers and trapped if you loaded a pointer with an invalid segment. And so loading the pointer into a register to compare it would crash.
Using a freed pointer is incorrect behavior, a bug in shorter terms. If you do anything with a freed pointer (other than assigning new memory), you're inviting all kinds of bugs (independent of what the compiler might be doing with your code).
I can't tell you exactly why but it's consistent with just about everything else involving p being undefined, and the result of the comparison would be useless anyway.
I can imagine situations where a pointer q might sometimes be a copy of pointer p and sometimes might point to something else, and the code wants to free q if and only if it is not a copy of p (because p has been free'd earlier).
Because a new object can have the same address as p, so comparing to p isn't enough to tell you if you have a copy of p or a live pointer to something else.
tl;dr `realloc(p, 0)` is slated to be undefined behavior in C23, whereas it's been somewhat implementation defined until now, with recommendation being realloc(p, 0) is equivalent to free(p)
Seems a bit tone deaf to create new undefined behavior in memory handling, especially when a sane default behavior seems to be de facto
I've used that free-on-0 behavior myself. Unfortunately the code that uses this will often have 0 be a length variable, so hard to grep for this. Ideally musl/glibc will both stick to that undefined behavior being free & gcc/clang won't go about making this something to point their optimizations at
Lest we have to stop using realloc outside of a safe_realloc wrapper
What got this whole thing weird is that C doesn't like zero sized objects, but implementations were allowed to return a unique pointer for a zero sized allocation. Which then raises the matter that being portable there require freeing that reserved chunk for non-free implementations. In theory this reservation code could be more efficient when code frequently reallocates between 0 & some small value. & there was uncertainty because NULL is a way to say allocation failure, but then if one did a NULL check on realloc's return value they also had to check that the size was non-zero
> Seems a bit tone deaf to create new undefined behavior in memory handling,
It's only tone deaf to people who understand "undefined behavior" as an epithet or as synonymous with giving a license to compilers to screw you over. The term doesn't have either of those meaning to those on the C committee. In fact, one of the explicit rationales for the proposal is that, "Classifying a call to realloc with a size of 0 as undefined behavior would allow POSIX to define the
otherwise undefined behavior however they please." https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2464.pdf
> especially when a sane default behavior seems to be de facto
The above proposal, N2464, gives the behavior for AIX, zOS, BSD (unspecified), MSVC (crt unspecified), and glibc. They each have different behaviors.
Why they chose to finally make it undefined (it was marked as obsolescent for a long time) rather than keep it as implementation-defined, I don't know. Perhaps because it 1) simplifies the standard, and 2) by making it undefined it suggests compilers should start warning about it--despite all this time neither has there arisen a consensus among implementations about the best behavior, nor are programmers aware that the behavior actually varies widely.
EDIT: The draft SUSv5/POSIX-202x standard has indeed directly addressed this issue. See, e.g., https://www.austingroupbugs.net/view.php?id=374 The most recent draft included the following addition to RETURN VALUE:
OB If size is 0,
OB CX or either nelem or elsize is 0,
OB either:
OB * A null pointer shall be returned
OB CX and, if ptr is not a null pointer, errno shall be set to [EINVAL].
OB * A pointer to the allocated space shall be returned, and the memory object pointed to by ptr
shall be freed. The application shall ensure that the pointer is not used to access an object.
CX marks points of divergence with C17. The first CX is because of the addition of reallocarray, absent from C17. The second is because POSIX will mandate the setting of EINVAL if NULL is returned.
>It's only tone deaf to people who understand "undefined behavior" as an epithet or as synonymous with giving a license to compilers to screw you over. The term doesn't have either of those meaning to those on the C committee.
It's unfortunate but not surprising that the C committee isn't aware of the problems with the undefined behavior.
In fact, after I started reading WG14 meetings minutes, I completely lost faith that any of the serious problems with the standard will ever get fixed.
This is not a problem with the committee and is not a problem with compiler writers. The committee is only marking certain behaviors as UB. Compilers can do what they think is more sensible in these situations. And compiler writers are not forcing you to accept these extreme optimizations. You always have the option of disabling optimizations and accept that your code has bugs (UB). You just need to test the code you write under different compiler settings, similarly to how you test code in different environments.
"just disable optimizations" is not a solution unless the compiler allows enough fine grained control where that solution is `-ffree-zero-sized-realloc`
> It's only tone deaf to people who understand "undefined behavior" as an epithet or as synonymous with giving a license to compilers to screw you over.
Unfortunately, this is the correct understanding of UB.
realloc to 0 size being free is useful in particular because it means a function pointer to realloc is a complete memory allocator: call realloc with pointer NULL to get malloc, and call realloc with size 0 to get free.
> What got this whole thing weird is that C doesn't like zero sized objects, but implementations were allowed to return a unique pointer for a zero sized allocation.
Some of the windows API's work like this, so how much is pressure from MS?
Pattern matching ram for variables/objects whilst they exist even if zero'ed or prefilled with a value doesnt give perfect security. Random values would make it harder to work out the variable/object.
There is no alternative to network protocols and IPC that the stringtypes C has. You get a length and a byte array. If you trust the user, you can assume length is correct. Otherwise no.
> Sure there are, as proven by distributed networking stacks not written in C.
this has nothing to do with the C language, but the structure of information. If the datatype contains a length, it has to be serialized anyway. There is no way of fixing this.
We have "pointer provenance" which allows license to track type punning across more of your program than ever before in order to delete more parts of it with no diagnostic required.
For bonus marks, int and atomic_int are unrelated types, and simd vector types aren't a thing, so enjoy the unfixable performance cost of choosing C.
I wonder if somewhere along the chain there was an automated tool to convert frequently abused mental-health related terms like “insane” into something leas hurtful, or something along those lines?
I haven’t seen widespread use of the word “neurodivergent” as a kind of… whatever this is, weirdly euphemistic slur, almost?
I haven’t seen widespread use of the word “neurodivergent” as a kind of… whatever this is, weirdly euphemistic slur, almost?
It’s a continuation of the euphemism treadmill [1]. It won’t be long before “neurodivergent” is considered politically incorrect and a new term is invented to replace it.
> I wonder if somewhere along the chain there was an automated tool to convert frequently abused mental-health related terms like “insane” into something leas hurtful, or something along those lines?
And yet again, these Lincoln systems mess up. While giving advice to the author can avoid Great Grimsby mistakes, making the replacements automatically is an utterly Scunthorpe decision, with failures as Slough as they are foreseeable.
If the intent is to be unhurtful, it should need to choose a different word to if the intent is to be hurtful! Even our most sophisticated automated systems are Milton Keynes at determining that kind of thing.
As an aside, this is one of the spots where GPT does a really good job of fixing things.
Reword the following passage. Change euphemisms to wording that has similar meaning though no negative conotations. Indicate changed words by putting them in "{{word}}"
###
Standards are supposed to lead
...
Why are such requests made? Often because of arithmetic bugs. And what is a non-null pointer from malloc(0) good for? Absolutely nothing, except shooting yourself in the foot.
###
To which it returned:
Standards are meant to provide the groundwork for a better world by enabling code portability. Real standardization necessitates {{managing}} a variety of compiler and library implementations while maintaining sensible rules. The saga of the realloc story demonstrates that C standardization does not operate in this fashion any longer.
As C89 was being crafted, there was a {{unconventional}} thought of a "zero-length object": Supporters proposed that a non-null pointer to such an object should be given for requests for zero-byte allocation.
Why do such requests come about? Frequently due to miscalculations. And what is a non-null pointer from malloc(0) useful for? Nothing, apart from {{setting oneself up for failure}}.
It changed (along with some wording around to make the changes fit):
heading cats -> managing
neurodivergent -> unconventional
shooting yourself in the foot -> setting oneself up for failure
I guess that's be the way to detect if the text has been written by the AI - it'd be completely devoid of metaphors and cleansed from anything that could possibly offend somebody. I wouldn't ever call it a "good job" but I guess it's useful.
“Unconventional” seems like a bad pick to me, too neutral. Clearly the author intends to say something negative about zero-length objects. And of course it is fine dislike things, it is just a matter of not using hurtful language.
There is no possible way to have style without the potential to bother someone. Just write how you feel. If the readers are so offended, they can stop reading. Life will go on.
> Reword the following passage. Change euphemisms to wording that has similar meaning though no negative conotations. Indicate changed words by putting them in "{{word}}"
>
> The couples were of unconventional make up, including male and female pairings, male and male pairings as well as female and female.
>> The couples had non-traditional compositions, with pairings consisting of men and women, men and men, and women and women.
Thanks for pointing this out---when I read the article I tripped on that word, thought it odd and not sure what the author was trying to say, and moved on, but now that you call it out it seems very obviously to be used in just the same way that a lot of people used to use the r-slur (and some still do).
It's still apt, even as someone ostensibly in that category.
It does require some abstract thinking to comprehend sets of zero measure, negative measure or complex measure in mathematics. A "zero length object" is also encountered pretty often in practice:http://docs.autodesk.com/CIV3D/2013/ENU/index.html?url=files... and zero-length files come to mind.
The euphemism ends up working out fine, though likely not the author's intent.
I don't agree with that description at all. Here's the code:
The only code path that's "entirely different" is lines 1,4,5 and in that case of course you remove a return that's after a return.And the other valid code path is 1,2,5, which has `puts` after `unreachable`.
To need `puts` you have to imagine a code path that gets past the "if" without taking either branch?
Maybe the author means something by "code path" that's very different from how I interpret it?
I would be pretty surprised if the above code means something different from: