[stochastician]

This is an incredibly interesting question, and the answer is an emphatic YES! Many systems involve iterative schemes, where the output of one step is used as the input to the next step. Here, these precision errors can accumulate, and if there's a multiplicative term in your equations, they can explode!

These sorts of problems are actually very common in a lot of scientific computing and simulation contexts, which is why many in the scientific computing community look aghast at the rise of FP16 (and even fp32) in machine learning applications. Of course, those algorithms are often of a _very_ different nature from (say) the large-scale linear algebra or PDE solvers we're using, but still it's pretty shocking if you're used to worrying about machine precision!

[mjevans]

Machine learning might also prefer focusing on the magnitude rather than an exact value. (With the lower precision number part more about being nudged between magnitude bins.) E.G. bfloat16

https://en.wikipedia.org/wiki/Bfloat16_floating-point_format