Hacker News new | ask | show | jobs
by lifthrasiir 3803 days ago
Hmm, just out of curiosity, would it be possible to tweak Grisu to use the same error analysis as Errol? While possibly slower in the modern architecture, it might result in the compact, fast enough and hassle-free algorithm that is good for most [1] purposes.

[1] I have seen some performance-oriented JSON encoders using Grisu2 instead of Grisu3 in favor of performance. They may be willing to switch to Errol3 in spite of such configuration problems...

1 comments

That really depends on what you are trying to get out of the algorithm. Using 64-bit integers, I don't think you can turn Grisu2 into an failure-free algorithm. However, you can definitely improve its reliability by handling special casing small integers (as is done in Errol2/Errol3). By my estimates, this should improve Grisu's success rate from ~99.5% to ~99.95% [see math at end]. The remaining 0.05% is far too large to for enumeration and using a lookup.

If you are willing to print numbers that are correct but possibly suboptimal, you can also use Errol2 and modify it to remove checking. This will give you the speed of Errol3 without having a lookup table. I might look at making such an implementation and maybe call it Errol2b.

[math for calculating approximate error rate] With an input floating-point format with p bits of precision and an intermediate format with q-bits of precision: the "chance" of value failing is close to 1 in 2^(q-p). With 2^p values per exponent, you're going to see about 2^(2p-q) errors per exponent. In the case if Grisu, q is 64 and p is 53, for a total of about 2^42 errors. These numbers only give rough estimates, you'd have to verify them manually.