[mort96]

The scary thing IMO is: your code might be fine with unsafe math optimisations, but maybe you're using a library which is written to do operations in a certain order to minimise numerical error, and unsafe math operations changes the code which are mathematically equivalent but which results in many orders of magnitude more numerical error. It's probably fine most of the time, but it's kinda scary.

[dahart]

It shouldn’t be scary. Any library that is sensitive to order of operations will hopefully have a big fat warning on it. And it can be compiled separately with fast-math disabled. I don’t know of any such libraries off the top of my head, and it’s quite rare to find situations that result in orders of magnitude more error, though I grant you it can happen, and it can be contrived pretty easily.

[planede]

You can't fully disable fast-math per-library, moreover a library compiled with fast-math might also introduce inaccuracies in a seemingly unrelated library or application code in the same executable. The reason is that fast-math enables some dynamic initialization of the library that changes the floating point environment in some ways.

[dahart]

> You can’t fully disable fast-math per library

Can you elaborate? What fast-math can sneak into a library that disabled fast-math at compile time?

> fast-math enables some dynamic initialization of the library that changes the floating point environment in some ways.

I wasn’t aware of this, I would love to see some documentation discussing exactly what happens, can you send a link?

[mort96]

> Can you elaborate? What fast-math can sneak into a library that disabled fast-math at compile time?

A lot of library code is in headers (especially in C++!). The code in headers is compiled by your compiler using your compile options.

[dahart]

Ah, of course, very good point. A header-only library doesn’t have separate compile options. This is a great reason for a float-sensitive library to not be header-only, right?

[mort96]

It's not just about being header-only, lots of libraries which aren't header-only still have code in headers. The library may choose to put certain functions in headers for performance reasons (to let compiler inline them), or, in C++, function templates and class templates generally have to be in headers.

But yeah, it's probably a good idea to not put code which breaks under -ffast-math in headers if possible.

[jcranmer]

https://github.com/llvm/llvm-project/issues/57589

Turn on fast-math, it flips the FTZ/DAZ bit for the entire application. Even if you turned it on for just a shared library!

[account42]

That's only one small part of -ffast-math/-Ofast though and not a very scary one at that.

[mort96]

But it's an example of -ffast-math affecting separately compiled libraries.

[londons_explore]

you're gonna hive to give us a concrete real world example to convince most of us...

[mort96]

I don't typically thoroughly read through the documentation for all the dependencies which my dependencies are using.

But you're correct that it's probably usually fine in practice.

[dahart]

That’s fair. Ideally transitive dependencies should be completely hidden from you. Hopefully the author of the library you include directly has heeded the instructions of libraries they depend on.

Hey I grant and acknowledge that using fast-math carries a little risk of surprises, we don’t necessarily need to try to think of corner cases. I’m mostly pushing back a little because using floats at all carries almost as much risk. A lot of people seem to use floats without knowing how inaccurate floats are, and a lot of people aren’t doing precision analysis or handling the exceptional cases… and don’t really need to.

[ska]

> A lot of people seem to use floats without knowing how inaccurate floats are,

Small nit, but floats aren't inaccurate, they have non uniform precision. Some float operations can be inaccurate, but that's rather path dependent...

One problem with -ffast-math is that a) it sounds appealing and b) people don't understand floats, so lots of people turn it on without understanding what it does, and that can introduce subtle problems in code they didn't write.

Sometimes in computational code it makes sense e.g. to get rid of denorms, but a very small fraction of programmers understand this properly, or ever will.

I wish they had named it something scary sounding.

[account42]

> Sometimes in computational code it makes sense e.g. to get rid of denorms, but a very small fraction of programmers understand this properly, or ever will.

"Some times" here being almost all the time. It is rare that your code will break without denormals if it doesn't already have precision problems with them.

[dahart]

I am talking about float operations, of course. And they’re all inaccurate, generally speaking, because they round. Fast math rounding error is not much larger than rounding error without fast mast.