Hacker News new | ask | show | jobs
by zokier 1600 days ago
If you want to move beyond standard floats, this paper could be good starting point: https://dl.acm.org/doi/pdf/10.1145/3385412.3386037

Right now 0.1+0.2-0.3 results = 2.775557562 \cdot 10^{-17}, which depending on your viewpoint is either expected or unexpected. Also (0.1+0.2)-0.3 is simplified to 0.1+0.2-0.3 for display, but results a different value = 5.551115123 \cdot 10^{-17}

1 comments

Wow, thanks. This is way more complex than I expected. When I started working on this I thought that I might want to switch to "binary coded decimal"[0] to avoid issues with inputs like (0.1 + 0.2 - 0.3).

I haven't had a chance yet to read the full paper, but it talks about fixing trig issues too. Would this fix my (unfortunate) result for `e^(ipi) + 1` to be exactly 0, too (instead of something ugly like (-210^-16 + j1.22*10^-16)).

And is there an implementation I could use?

At some point I plan on open sourcing this code, so if I never manage to fix this then maybe someone else might someday do it. Originally I figured I could finish this whole project in a few months, especially after figuring out webassembly and discovering MathJax. But it ended up taking way longer than that, with little things constantly popping up that I wanted to fix before I could even think about sharing it.

[0]: https://en.wikipedia.org/wiki/Binary-coded_decimal

> When I started working on this I thought that I might want to switch to "binary coded decimal"[0] to avoid issues with inputs like (0.1 + 0.2 - 0.3).

If you want to go this route, there are standardized decimal floating points, e.g. https://en.wikipedia.org/wiki/Decimal64_floating-point_forma... (or the 128 bit version).

> And is there an implementation I could use?

Afaik Boehms implementation lived only in the Android source tree, i.e. https://android.googlesource.com/platform/packages/apps/Exac... . So to use that you'd need to extract and port them, which might be some effort.

There are all sorts of different approaches for doing calculations on real numbers. I think Johanssons work is particularly impressive in this domain, but I'm not sure if that is much use for a calculator like this: https://fredrikj.net/