That may be true as long as you use the same compiler versions as the developer used when writing/testing their code.
But then someone decides to deploy that code on an (old and) stable version of Linux with old(er) GCC version and things break.
For example ClamAV used to break quite often when built with older compiler versions, in the end we had to add runtime tests to configure based on testcases from the GCC bugzilla to make sure people are not using a "stable" and broken compiler. GCC 4.1.0 and 4.1.1 were particularly bad versions
You might think that you could still use -O2 and selectively turn off optimizations that are known to cause trouble (i.e. -fno-strict-aliasing), but there are cases where turning off that optimization exposes bugs in other optimization passes and a compiler that would be otherwise good at -O2 would break at "-O2 -fno-strict-aliasing":
https://bugs.gentoo.org/show_bug.cgi?id=275928#c29https://bugzilla.clamav.net/show_bug.cgi?id=1581