Hacker News new | ask | show | jobs
by vlovich123 1095 days ago
I think it’s just about Intel’s intermediate 80bit precision. Anything else?
3 comments

There are lots of different ways floating point determinism can get lost. The obvious one, as you mentioned, is the 80 bit x87 unit. Lots of 32 bit compilers will compile to doing floating point math on the x87, which is slow, but the same compiler compiling in 64 bit mode will use SSE2 instructions.

Floating point arithmetic is not associative. That is, (a+b)+c != a+(b+c) and (a*b)*c != a*(b*\c).

Multiplying by the reciprocal is not equal to dividing by the value. That is, x*(1/y) != x/y. An optimizing compiler may, when you attempt to divide by a constant, will optimize that to multiplying the reciprocal instead, that is, if you have code that divides by the constant 3 it will multiply by 0.33333333333333333, because it's a lot faster.

FMA (fused multiply-add) instructions are more precise and therefore not equal to the same calculation without FMA. That is, a*b+c != a*b+c if one compiler will output FMA instructions and the other one does not. (this will be true even in the same compiler with different flags)

Special functions are fucky. sqrt, sin, cos etc might not always give equal values. Or even in the same compiler if minor alterations to the code are made. A compiler might use one algorithm to compute sin(x), but a different algorithm if it needs to compute both sin(x) and cos(x) at the same time.

Floating point rounding mode is a thing. Sometimes a plugin changes your floating point rounding mode. Sometimes this plugin will be inserted into your runtime without your knowledge, such as an antivirus program, or malware that hijacks your browser to give "better"/"customized" shopping/search recommendations. There was a bug writeup about a crash in Chrome several years back, but I can't find it.

Basically you should assume that floating point math is non-deterministic. If you think you need deterministic floating point math, try to reformulate the problem so that you don't need deterministic floating point math. If you really* need deterministic floating point math, understand that you're signing up for a lot of pain.

AFAIK, the first three things you listed would be deterministic across compilers unless you enabled -ffast-math (which isn't what this is talking about) precisely for that reason. I believe the same applies to intrinsics and rounding modes but not sure, especially since the website talks about GCC vs clang differences for the latter.

[1] https://stackoverflow.com/questions/55974090/clang-gcc-only-...

Any reorderings or optimizations of floating point code could potentially change the results even without 80-bit precision active.
AFAIK only if you have -ffast-math enabled. Otherwise IEE754 very specifically lays out all the optimizations the compiler is and isn't allowed to do.
Constant folding of floats may depend on the compiler of compiler (e.g. 0.5 + x + 0.6 + 0.7), the compiler may be non-deterministic (std::unordered_map<float> constants), and it may depend on the CPU your compiler is running on.