[AshamedCaptain]

> Often this comes down to acceptable implementation defined behavior,

I believe this is "always" rather than often when it comes to the actual operations defined by the FP standard. gcc does play it fast and loose (as -ffast-math is not yet enabled by default, and FMA on the other hand is), but this is technically illegal and at least can be easily configured to be in standards-compliant mode.

I think the bigger problem comes from what is _not_ documented by the standard. E.g. transcendental functions. A program calling plain old sqrt(x) can find itself behaving differently _even between different stepping of the same core_, not to mention that there are well-known differences between AMD vs Intel. This is all using the same binary.

[mabster]

I'm surprised by this, regarding sqrt. The standard stipulates correct rounding for simple arithmetic, including sqrt ever since 754 1985.

Unless of course we are talking about the 80 bit format.

If that's not the case, would be interested to know where they differ.

Unfortunately for the transcendental function the accuracy still hasn't been pinned down, especially since that's still an ongoing research problem.

There's been some great strides in figuring out the worst cases for binary floating point up to doubles so hopefully an upcoming standard will stipulate 0.5 ULP for transcendentals. But decimal floating point still has a long way to go.

[AshamedCaptain]

Because compilers can and have implemented sqrt in terms of rsqrt which is .. fun to work with. This also on SSE.

[mabster]

I spent most of my career working with rsqrt haha. And had my fair share of non-754 architectures too!

Every 754 architecture (including SSE) I've worked on has an accurate sqrt().

I'm assuming you're talking about with "fast math" enabled? In which case all bets are off anyway!

[AshamedCaptain]

No; compilers have done this even without fast-math. Gcc does not seem to do this anymore, but still does plenty of unsafe optimizations by default, like FMA.

Or maybe the library you use...

[mabster]

Argh, sounds really frustrating! It's hard enough to get accuracy when you can control operations never mind when the compiler is doing magic behind the scenes!

FMAs were difficult. The Visual Studio compiler in particular didn't support purposeful FMAs for SSE instructions so you had to rely on the compiler to recognise and replace multiply-additions. Generally I want FMAs because they're more accurate but I want to control where they go.

[jcranmer]

sqrt is a fundamental IEEE 754 operation, required to be correctly rounded, and many architectures implement a dedicated, correctly rounded sqrt instruction.

Now, there is also often an approximate rsqrt and approximate reciprocal, with varying degrees of accuracy, and that can be "fun."