[jhgb]

> I've found algorithmic (I prefer this to "automatic") differentiation

Isn't every differentiation algorithmic? It's typically integration where you often use intuition in substitution, unless you're using something like Risch's algorithm, but one learns to differentiate in school from the very beginning with what can only be described as an algorithm: at every step, there's a clear procedure to decide on what to do next, and when to terminate.

[omnicognate]

There are 3 ways of doing differentiation in a computer: numeric, X and symbolic, where X is the technique we're discussing. Personally I don't think "automatic" really captures what's different about this technique. It's no more "automatic" than the other options.

"Algorithmic" differentiation is intended to evoke the idea that we are "differentiating the algorithm" piece by piece, although it's certainly true that it can be misunderstood as saying that the differentiation process itself is an algorithm, which is again no different to the other techniques.

It's a shame the terminology in this area is so fragmented. I assume it's because the technique has been rediscovered many times. In machine learning, for example, what I would call "adjoint algorithmic differentiation" is called "backpropagation".

[jhgb]

Ah, I see what you mean, but if you contrast "automatic differentiation" and "algorithmic differentiation" with each other, it's counterintuitive to me that these terms should be "oriented" differently. Either I imagine that the differentiation is automated or algorithmized, or I could imagine that it's an automaton or algorithm being differentiated. But it's confusing to me if it's algorithm being differentiated vs. differentiation being automated. Not sure if that explains sufficiently clearly what confused me.