[sundarurfriend]

> We came to the conclusion that a global fastmath option is impossible to use correctly in Julia.

I'd assumed that global fastmath was a bad idea in general, and assumed that was the reason for making this a no-op. Is there a reason it's particularly bad in Julia, some assumptions the standard library makes or something?

[adgjlsfhk1]

It is bad in general, but it ends up being worse in Julia because C and C++ generally aren't compiled with whole program optimization. global fastmath is more aggressive the more you inline, and in C/C++ the math library is usually a statically linked library which creates an inlining barrier. Julia has all the code at runtime, and therefore is often able to run faster by inlining more code. The downside of this is that a global fastmath flags will optimize more than you think they should and give even more wrong answers than usual.

[xmcqdpt2]

From the example in the article (exp) it sounds like they are implementing highly optimized versions of transcendental functions. This is great! One of the reason gfortran is so much slower than the intel fortran compiler is the slow special functions it uses. However those tricks appear to degrade badly under some LLVM optimizations that are enabled with fastmath.

It makes sense to optimize for the non-fast math case because that's the recommended setting, and I guess having two implementations of all the (very important, easy to mess up, core) special functions + all the testing infra to check that they work correctly on all platforms was probably deemed too much work for marginal benefits.

[NeuroCoder]

From a non-technical point of view (since the technical answer was already provided) I think this sort of magic optimizations is a double edge sword in any language. No matter what you do there will be corner cases that need manual tuning that become inaccessible behind some init option.

[sundarurfriend]

That's especially true for `fastmath`, which shoves a bunch of optimization tradeoffs together, some of them "handle with care" territory, some of them in the "faulty live grenade that'll probably explode in your face" territory.

I was just wondering why the post said "impossible to use correctly in Julia" rather than just "impossible to use correctly", but writing it out now, I realize that "impossible" would be hyperbole for the latter, it would be more like "highly likely to go wrong".

[ChrisRackauckas]

fastmath is bad in general. In Julia it's not as bad as many other languages because it's attempted to be kept local. The @fastmath macro is essentially a find-replace macro, where it finds things like ^ and replaces it with the fast_pow function which drops the 1ulp requirement. However, this global option was really the one piece left in Julia that where it could creep in, hence the reason to drop it.

There are still some other difficulties of course, since fastmath in the C ABI is quite wild (or I guess, it's really the GCC implementation up to GCC 13 (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55522#c45)). Simon wrote a nice piece about the difficulties in general: https://simonbyrne.github.io/notes/fastmath/. In a general sense there is still the potential vulnerability that effects the Python ecosystem which is that if any package has binaries built with fastmath it could cause other calculations to be fastmath as well in a non-local way (https://moyix.blogspot.com/2022/09/someones-been-messing-wit...):

> It turns out (somewhat insanely) that when -ffast-math is enabled, the compiler will link in a constructor that sets the FTZ/DAZ flags whenever the library is loaded — even on shared libraries, which means that any application that loads that library will have its floating point behavior changed for the whole process. And -Ofast, which sounds appealingly like a "make my program go fast" flag, automatically enables -ffast-math, so some projects may unwittingly turn it on without realizing the implications.

With Julia, there is the advantage here that (a) most libraries don't have binary artifacts being built in another language, (b) the Julia core math library is written in Julia and is thus not a shared library effected by this, and (c) those that do have their binaries built and hosted in https://github.com/JuliaPackaging/Yggdrasil. So in the binary building and delivery system you can see there are some patches that forcibly remove fastmath from the binaries being built to avoid this problem (https://github.com/search?q=repo%3AJuliaPackaging%2FYggdrasi...). The part (b) of course is the part that is then made globally safe by the removal of the flag in Julia itself, so generally Julia should be well-guarded against this kind of issue with these sets of safeguards in place.