[goalieca]

Floating point arithmetic is neither commutative or associative so you shouldn’t.

[lo0dot0]

While it technically correct to say this it also gets the wrong point across because it leaves out the fact that ordering changes create only a small difference. Other examples where arithmetic is not commutative, e.g. matrix multiplication , can create much larger differences.

[kstrauser]

> ordering changes create only a small difference.

That can’t be assumed.

You can easily fall into a situation like:

  total = large_float_value
  for _ in range(1_000_000_000):
    total += .01
  assert total == large_float_value

Without knowing the specific situation, it’s impossible to say whether that’s a tolerably small difference.

[StefanKarpinski]

Floating-point arithmetic is non-associative, but it is commutative for the operations that are algebraically commutative: x + y == y + x and x*y == y*x. And x - y = -(y - x) so subtraction is properly anti-commutative.

The only very marginal exception to this is that when both arguments are NaN, the return value will be NaN, but which NaN payload is returned can depend on argument order. But no one ever uses this because it's not specified, so it can't be used reliably for anything useful. The behavior I wish IEEE 754 had specified for this is to define a standard NaN value (or two), and when the return value of an op is NaN, and some of the arguments are non-standard NaNs, then one of those non-standard NaN values must be returned. This doesn't depend on argument order and allows NaN payloads to be reliably propagated, which would let you encode useful debugging information in NaN payloads and know that it will flow through the program.

[layer8]

IEEE-754 addition and multiplication is commutative. It isn't distributive, though.

[eapriv]

Why is it not commutative?

[layer8]

It actually is commutative according to IEEE-754, except that in the case of a NaN result you might get a different NaN representation.

[adgjlsfhk1]

having multiple NaNs and no spec for how they should behave feels like such an unforced error to me

[layer8]

For mathematical use, NaN payloads shouldn’t matter, and behave identically (aside from quiet vs. signaling NaNs). It also doesn’t matter for equality comparison, because NaNs always compare unequal.

[adgjlsfhk1]

from the user perspective it's not too bad, but from the compiler perspective it is. The result of this is that LLVM has decided that trying to figure out which nan you got (e.g. by casting to an Int and comparing) is UB, which means pretty much every floating point operation becomes non-deterministic.

This also adds extra complexity to the CPU. you need special hardware for == rather than just using the perfectly good integer unit, and every fpu operation needs to devote a bunch of transistors to handling this nonsense that buys the user absolutely nothing.

there are definitely things to criticize about the design of Posits, but the thing they 100% get right is having a single NaN and sane ordering semantics