Hacker News new | ask | show | jobs
by zaroth 4620 days ago
As very much not a C programmer, I did sort of grasp the argument that while the overflow behavior may be undefined, once the overflow happens and if the end result is that the sign goes negative, how can you reasonably skip the negative branch?

  int a,b,c;
  a=INT_MAX;     /* statement ONE */
  b=a+2;         /* statement TWO */
  c=(b>a);       /* statement THREE */
1 comments

Undefined behavior means anything is allowed.

If you have a buffer overflow which overwrites your x86 instructions to completely change the behavior of the program, that's "OK" because buffer overflows are "undefined behavior". Simple cause and effect.

The authors of compiler optimizers noted this basically gave them free reign to optimize based on the assumption that undefined behavior does not happen, because if it DID happen, whatever the program ends up doing in that case of not a concern, as whatever it is, it's "OK".

An example of this: If I dereference a null pointer and try and assign it to three, and then after that dereference check if the pointer is null, I can optimize away that check.

On a typical x86 system, this sounds reasonable: "It'll crash anyways, right?" ...however, the optimizer is allowed to do this even if there's no guarantee there will be an access violation because I'm using an unprotected memory model, or the member is a sufficient way into the containing struct to be in a seperate mapped page, etc.

Taken to a further extreme, this applies to signed integer overflow as well. "You're not allowed to do it within a well defined program, ergo, we assume you don't do it and optimize away based on that assumption."