| Let’s say you wrote a function f(x) and want to know where it is zero. The traditional way is Newton’s method (https://en.wikipedia.org/wiki/Newton's_method) That requires you to compute f’(x) at various points. You could either pick some small epsilon and approximate it as (f(x+ε)-f(x-ε))/2ε or implement a function that symbolically computes it (likely returning better approximations). This kind of stuff gives you the latter for free. That’s less work (more so if you’re continuously tweaking the definition of f(x)) and less error prone (once the bugs have been ironed out :-) ). Now, you’ll ask “who ever wants to compute zeroes?”. Turns out that’s basically (1) what machine learning does. It searches for a maximum, but that can be done using Newton’s method (https://en.wikipedia.org/wiki/Newton's_method#Minimization_a...) (One problem that these kinds of approaches overlook is that of numerical stability. I fear many uses will see implementers help their automatic differentiation tooling to improve numerical stability) (1) yes, I’m painting in very broad strokes, losing detail. |