Hacker News new | ask | show | jobs
by nwhitehead 3811 days ago
This example makes it really clear what's going on. Could someone translate it to do reverse automatic differentiation? That's the one I never quite understand.
2 comments

You mean integration? That's much harder, but has been done automatically. Symbolic differentiation is easy, because you can just keep blindly applying a set of rewrite rules until none of them apply. That process converges on a unique result. Symbolic integration doesn't converge in that way. More strategy is required, and you're not guaranteed a closed form solution. Mathematica has a good symbolic integrator.
Symbolic integration doesn't converge in that way. More strategy is required, and you're not guaranteed a closed form solution.

However, if a closed-form solution exists which can be expressed in terms of the operations + - * / exp log, then it is guaranteed to be found.

Reverse mode AD is another algorithm for calculating derivatives

    https://en.wikipedia.org/wiki/Automatic_differentiation#Reverse_accumulation
The short answer is no, for a couple of reasons. First, you have to distinguish whether you want the antiderivative or the integral. In the first case, the problem with finding the antiderivative is that it's not unique. For example, the antiderivative of x can be 0.5 * x^2 or 0.5 * x^2 + 1 or 0.5 * x^2 + 2, etc. Basically, taking derivatives can lose information and we don't know how to put that information back. In the second case, we have the integral, but that creates problems because the integral really needs to have a domain defined to make sense. Basically, over what values are we integrating? In 1-D we can specify this with a bunch of intervals, but in 2 and higher dimensions, it becomes difficult. In fact, this is partially why we have meshing tools like Cubit or gmesh. Yes, we use them with finite element methods, but really, at it's core, we need them to break up a domain because we want to integrate the pieces.

Anyway, with enough information or on a simple enough setup, both can actually be done, but the general case isn't super well defined, which is why the simple answer is no.

I read the parent (as does 'tome in a neighboring thread) to be asking about reverse-mode automatic differentiation (where what is explained in TFA is forward-mode), not about computing antiderivatives.
Well, crap, you're right. The other comment screwed me up. Apologies!