[ChrisRackauckas]

The Julia package ecosystem has a lot of safeguards against silent incorrect behavior like this. For example, if you try to add a package binary build which would use fast math flags, it will throw an error and tell you to repent:

https://github.com/JuliaPackaging/BinaryBuilderBase.jl/blob/...

In user codes you can do `@fastmath`, but it's at the semantic level so it will change `sin` to `sin_fast` but not recurse down into other people's functions, because at that point you're just asking for trouble. There's also calls to rename it `@unsafemath` in Julia, just to make it explicit. In summary, "Fastmath" is overused and many times people actually want other optimizations (automatic FMA), and people really need to stop throwing global changes around willy-nilly, and programming languages need to force people to avoid such global issues both semantically and within its package ecosystems norms.

[aidenn0]

Automatic FMA can change the result of operations, so it makes (some) sense to be bundled in with fastmath.

[ChrisRackauckas]

But if what you want is automatic FMA, then why carry along every other possible behavior with it? Just because you want FMA, suddenly NaNs are turned into Infs, subnormal numbers go to zero, handling of sin(x) at small values is inaccurate, etc? To me that's painting numerical handling in way too broad of strokes. FMA also only increases numerical accuracy, it doesn't decrease numerical accuracy, so bundling it with unsafe transformations makes one uncertain now whether it has improved or decreased accuracy.

For reference, to handle this well we use MuladdMacro.jl which is a semantic transformation that turns x*y+z into muladd expressions, and it does not recurse into functions so it does not change the definitions of the callers inside of the macro scope.

https://github.com/SciML/MuladdMacro.jl

This is something that will always increase performance and accuracy (performance because muladd in Julia is an FMA that is only applied if hardware FMA exists, effectively never resorting to a software FMA emulation) because it's targeted to do only a transformation that has that property.

[adgjlsfhk1]

"FMA also only increases numerical accuracy, it doesn't decrease numerical accuracy". This strictly speaking isn't true. For example, xx-yy will sometimes be more accurate than fma(x,x, -y*y) (e.g. if x==y). That said, I agree that fma is different from the rest in that it will on average, usually, increase accuracy.

[eigenspace]

This isn't really as valid a comparison as you might think it is. The results of operations varying is not the problem with 'fast-math', the problem is that can negatively impact accuracy in catastrophic ways (among other things).

Sure, automatic FMA can change the result, but to my knowledge it always gives a more accurate result, not a less accurate one, and the way in which the results may differ is bounded.

[adgjlsfhk1]

fma can give less accurate results, and can give very large differences. for example 1.5floatmax(Float64)-floatmax(Float64) is Inf, while the fma version will give .5floatmax(Float64)