That's a common misconception. Advanced compilers like GCC and LLVM have optimizers working primarily on a common intermediate representation of the program, which is largely backend-independent, and written for semantics of the C abstract machine.
UB has such inexplicable and hard to explain side effects, because all the implicit assumptions in the optimization passes, and symbolic execution used to simplify expressions follow semantics of the C spec, not the details of the specific target hardware.
Programmers have an ideal obvious translation of C programs to machine instructions in their head, but there's no spec for that.
It creates impossible expectations for the compilers. My recent favorite paradox is: in clang's optimizer comparing addresses of two variables always gives false, because they're obviously separate entities (the hardcoded assumption allows optimizing out the comparison, and doesn't stop variables from being in registers).
But then clang is expected to avoid redundant memcpys and remove useless copies, so the same memory location can be reused instead of copying, and then two variables on the stack can end up having the same address, contradicting the previous hardcoded result. You get a different result of the same expression depending on order of optimizations. Clang won't remove this paradox, because there are programs and benchmarks that rely on both of these.
And yet, in practice the C have written over the last 20+ years mostly ended up with "undefined behaviour" = "what CPU does"...
That may not be true in fact but it ends up that way because "undefined behaviour" tends to be implemented "to work" and we're used to behaviour of common CPUs that may have fed into expectation of what "to work" should be...
That's not how compiler developers interpret undefined behaviour. Undefined behaviour is closer to an 'U', 'X' or "don't care" in VHDL. These things don't exist in real hardware and only in simulation, therefore the synthesis tool simply assumes that they never happen and optimizes according to that. However, C does not have a simulation environment and UB propagation is not a thing. It will simply do weird shit, like run an infinite loop, because you forgot to write the return keyword, when you changed a function from void.
"undefined behavior behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements
NOTE Possible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message).
EXAMPLE An example of undefined behavior is the behavior on integer overflow." (The C99 standard, section 3.4.3)
translates into "whatever your CPU does" because while there is not requirement imposed, in general the compiler does make it work "in a manner characteristic of the environment".
I believe that memory accesses outside of array bounds, signed integer overflow, null pointer dereference are all examples of "undefined behaviour", which in practice all boil down to what the CPU does in those cases. I.e. commonly memory access outside of array bounds returns whatever is at that address as long as address is valid because there are no checks and that's what the CPU does when asked to load from address. Integer overflow? If a result of adding/subtracting, commonly it wraps around because that's how the CPU behaves, etc.
And I believe this is all on purpose. C is an abstraction over assembly and I believe that people who were used to their CPU's behaviour wanted to keep it that way in C, and also compilers were simple.
For someone who's been writing "C for 20+ years" according to your other post, you come across as extremely ignorant of how modern optimizing C compilers work. I suggest you thoroughly read and understand Russ Cox's simple expose [1] and the UB posts in John Regeh'r blog [2].
The C security issues that plague us today are partly fueled by the attitude of guesswork and ignorance demonstrated in your posts.
UB has such inexplicable and hard to explain side effects, because all the implicit assumptions in the optimization passes, and symbolic execution used to simplify expressions follow semantics of the C spec, not the details of the specific target hardware.