[sillysaurusx]

If you want to learn how to build autodiff from scratch in python, see autodidax: https://jax.readthedocs.io/en/latest/autodidax.html

It's light on math and heavy on simplicity. I've been studying it for about 9 months and I still learn new things.

[sritchie]

If you want to try a literate programming take on forward-mode automatic differentiation in Lisp (Clojure), check out the Differential implementation in my sicmutils library: https://github.com/sicmutils/sicmutils/blob/main/src/sicmuti...

Here's the org-mode export of the file, with all TeX rendered: https://samritchie.io/dual-numbers-and-automatic-differentia...

I worked on this with Gerald Sussman, and it has the somewhat unique feature of being able to take derivatives of higher-order functions. The system is also extensible, so this feature could have been installed outside of the core library, and in fact would need to be to support, say, functions that returns JS futures, scala functions etc.

Enjoy!