[cogman10]

Ehh, not so much inaccurate, more of a "floating point numbers are tricky, let's act like they aren't".

Compilers are pretty skittish about changing the order of floating point operations (for good reason) and ffast-math is the thing that lets them transform equations to try and generate faster code.

IE, instead of doing "n / 10" doing "n * 0.1". The issue, of course, being that things like 0.1 can't be perfectly represented with floats but 100 / 10 can be. So now you've introduced a tiny bit of error where it might not have existed.

[rwmj]

It isn't just that. -ffast-math also allows the compiler to ignore infinites. In fact for GCC with -ffast-math, isinf always returns false. Something similar happens for NaNs/isnan.

[cogman10]

I lump this into "floating points are tricky". NaNs and inf are definitely legitimate floating point values. They are also things that a lot of applications will break on they ever encounter them.

[phkahler]

I've never understood why generating exceptions is preferable to just using higher precision.

[gumby]

Higher precision isn’t always available. IEEE 754 is an unusually well-thought-through standard (thanks to some smart people with a lot of painful experience) and is pretty good at justifying its decisions, some of which are surprising (far from obvious) to anyone not steeped in it.

[cogman10]

The main problem with floats in general is they are designed primarily for scientific computing.

We are fortunately starting to see newer (well, not that new now) CPU instructions like FMA that make more accurate decimal representations not take such huge performance hits.

[adgjlsfhk1]

what does fma have to do with decimal numbers?

[cogman10]

Oh shoot, nvm, I thought it was an optimization for integers.

Really it'll be the SIMD style instructions that speeds things up.

[dahart]

On a GPU, higher precision can cost between 2 and 64 times more than single precision, with typical ratios for consumer cards being 16 or 32. Even on the CPU, fp64 workloads tend to run at half the speed on real data due to the extra bandwidth needed for higher precision.