[sbradford26]

If I have data going out on a data bus in grams/second, but someone reading that data in a different sub system reads it and thinks it is kilograms/second because that is the default unit they use then you have an issue. The actual conversion of the bits isn't the issue it is that certain systems might use different units internally and making sure those conversions are done correctly. It is much more an engineering design/human interaction issue than a computation issue.

[Merad]

Pretty sure GP was making a sarcastic comment referencing how floating point is notorious for subtle rounding issues when handling values that have an exact representation in base 10 but a repeating pattern in binary.

[fnordpiglet]

Yeah. Given we’ve moved from counting on our fingers to counting on our computers we need a new base 2 metric.

[fnordpiglet]

Yes, I understood that. However repeated conversions can lead to rounding errors such as catastrophic cancellation. This is even more esoteric than “grams here kilograms there” and thus easier to fall prey to. Depending on how precise your measurements are you could easily drop significant digits or add random noise, especially if you do large scale changes.

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.h...

[sbradford26]

Yeah we tended to utilize fixed point for many of our variables and documented the precision for each one. Then an analysis was done verifying that we were maintaining precision through the calculations from input from a sensor to output.

[fnordpiglet]

The problem is almost nothing today accelerates fixed point and everything supports floating point accelerations, which will speed up your ability to calculate the wrong number by many orders of magnitude.

[sbradford26]

Yeah that makes sense. In aerospace they will fall on the side of accuracy even at the expense of speed. But they also tend to utilize FPGAs that can be designed to handle fixed point calculations quickly but with that comes a lot of cost and specialty hardware.

[fnordpiglet]

Another place it makes issues is financial modeling, particularly for complex derivatives where the model might be extremely complex and small errors can accumulate into meaningful errors in risk calculations. Fixed point is sometimes used in finance but performance is also a real concern. There’s a lot you can do to reduce fp errors if your numerical libraries are carefully constructed. But it has often made me wonder why there aren’t processor lines with high performance fixed point since the math is extremely easy - even by just shifting the mantissa and working in integer space then shifting it back.