Hacker News new | ask | show | jobs
by AlotOfReading 19 days ago
As I said, floats can provide results that are no worse than a specified fixed point type. So if you want uniform absolute precision, just round down to the required precision.

Floating point is generally deterministic in practice with a fairly minor amount of effort, the major remaining issue being library rounding. I actually wrote a library that guarantees this for arbitrary code, with some small, obvious caveats like standard library precision. And the conference talks linked above note, the standard library issues are an increasingly solved problem for modern toolchains. The remaining cases are mostly things you won't do in fixed point. Let me know if you're aware of anyone computing erfc in fixed point for determinism though.

I'm not saying there aren't any situations where other systems are justified, but you probably won't know if you fall into any of them without the kind of numerical analysis that most codebases will never receive.

1 comments

Totally fair if you have full control, my experience is often with databases where you can get warnings that float implementations can even change per-operating system (thanks MySQL) or per query plan (based on plan ordering) which is ... pretty bad!
I don't know much about real query planners, but if I understand what you're saying there may be ways to improve the situation.

If you stick to the safe bits I've been discussing elsewhere in this thread and your platforms implement IEEE floats, float math will also be commutative + associative and you won't have to deal with precision loss. That means your usable range will be narrower than the same size fixed type (because it's limited by the mantissa) but it's large enough to still be useful.