Hacker News new | ask | show | jobs
by dvt 2184 days ago
This is an awesome article! One of my favorite SO answers I remember researching dealt with the Padé approximation of tanh[1] (which I found was significantly better than the Taylor expansion), but the caveat was that it only worked within a very narrow neighborhood.

I will say that the article didn't really touch on techniques that minimize IEEE 754 subnormal[2] performance impact, which is a very interesting problem in itself. A lot of real-life implementations of something like e^x will have various clever ways to avoid subnormals.

[1] https://stackoverflow.com/a/6118100/243613

[2] https://mathworld.wolfram.com/SubnormalNumber.html
3 comments
fractionalhare 2184 days ago
Hi, I am the author. You make a good point about subnormals, thank you. I will jot that down for a future update to the doc :)

It is funny you mention the Padé approximation; I was debating whether or not to include that in this article. Originally I planned to stop after Minimax (using Remez), but if I include rational approximation schemes anyway (e.g. Carathéodory-Fejer), it probably makes sense to implement and analyze that as well.

link
dia80 2184 days ago
Just jumping into to say I am loving the article, thanks! I'm dealing with a lot of exps in an optimization problem right now and frankly only have cursory understanding of what's going on under the hood so this was very enlightening for me. Thanks again.
link
raphlinus 2184 days ago
You might enjoy a few of my favorite sigmoids: https://raphlinus.github.io/audio/2018/09/05/sigmoid.html

That's not a deep exploration of the numerical analysis (especially compared with OP), but it does make one point: traditional "numerical analysis" techniques tend to assume the traditional four mathematical operators as canonical, but if your goal is to get something computed on actual computer hardware quickly and reasonably accurately, you have a wider palette to play with. In particular, inverse square root is a primitive about as efficient as reciprocal on modern hardware (or if using Carmack's wtf trick).

link
dvt 2184 days ago
This is great! Sometimes I wish I took more analysis back in school.
link
m4r35n357 2184 days ago
Just a suggestion, have you looked at the MPFR project, and derivatives: https://www.mpfr.org/

http://mpfr.loria.fr/mpfr-current/mpfr.html#index-mpfr_005fs...

It is way to deep in the grubby details for my pay grade, but who knows?

link