But if what you want is automatic FMA, then why carry along every other possible behavior with it? Just because you want FMA, suddenly NaNs are turned into Infs, subnormal numbers go to zero, handling of sin(x) at small values is inaccurate, etc? To me that's painting numerical handling in way too broad of strokes. FMA also only increases numerical accuracy, it doesn't decrease numerical accuracy, so bundling it with unsafe transformations makes one uncertain now whether it has improved or decreased accuracy.
For reference, to handle this well we use MuladdMacro.jl which is a semantic transformation that turns x*y+z into muladd expressions, and it does not recurse into functions so it does not change the definitions of the callers inside of the macro scope.
This is something that will always increase performance and accuracy (performance because muladd in Julia is an FMA that is only applied if hardware FMA exists, effectively never resorting to a software FMA emulation) because it's targeted to do only a transformation that has that property.
"FMA also only increases numerical accuracy, it doesn't decrease numerical accuracy". This strictly speaking isn't true. For example, xx-yy will sometimes be more accurate than fma(x,x, -y*y) (e.g. if x==y). That said, I agree that fma is different from the rest in that it will on average, usually, increase accuracy.
This isn't really as valid a comparison as you might think it is. The results of operations varying is not the problem with 'fast-math', the problem is that can negatively impact accuracy in catastrophic ways (among other things).
Sure, automatic FMA can change the result, but to my knowledge it always gives a more accurate result, not a less accurate one, and the way in which the results may differ is bounded.
fma can give less accurate results, and can give very large differences. for example 1.5floatmax(Float64)-floatmax(Float64) is Inf, while the fma version will give .5floatmax(Float64)
For reference, to handle this well we use MuladdMacro.jl which is a semantic transformation that turns x*y+z into muladd expressions, and it does not recurse into functions so it does not change the definitions of the callers inside of the macro scope.
https://github.com/SciML/MuladdMacro.jl
This is something that will always increase performance and accuracy (performance because muladd in Julia is an FMA that is only applied if hardware FMA exists, effectively never resorting to a software FMA emulation) because it's targeted to do only a transformation that has that property.