Hacker News new | ask | show | jobs
by brandmeyer 1936 days ago
This is actually exactly what I mean. Its probably the most common bug I've come across in this class. I don't expect unit tests to capture all rounding bugs (say, due to serialization and de-serialization through text). But I do expect to capture gross errors, such as an inadvertent cast to lower-precision somewhere in the pipeline.

I've worked with highly experienced and accomplished software engineers that expected interchange through protobuf or sql to be inaccurate due to rounding. No! If you stick a finite number in, you should get the exact same finite number back out again. Direct equality is fine for most cases. The sign bit of zero and NaN should also be returned faithfully and tested using memcmp when required.

IMO, the payload bits of NaN should also be also returned faithfully, but too many systems in common practice drop them.