[acidflask]

> A big problem is that chained floating point operations are a O(sqrt n) martingale.

Mostly false; see https://www.cs.berkeley.edu/~wkahan/Mindless.pdf

It is true that an arbitrary, independent sequence of n floating point operations can potentially lose up to 2n bits of precision. However, it is false that most scientific computing code has such structure. On the contrary, the loss of precision in numerical algorithms can be estimated quite well using standard techniques of error analysis, especially when there is underlying matrix structure to exploit. The growth of error in properly written numerical code should NOT grow exponentially with the number of floating point operations. It should be bounded by intrinsic properties of the underlying matrix, such as its condition number or spectral norm.