[rdyro]

A really cool post and a great set of visualizations!

Computing sparse Jacobians can save a lot of compute if there's a real lack of dependency between part of the input and the output. Discovering this automatically through coloring is very appealing.

Another alternative is to implement sparse rules for each operation yourself, but that often requires custom autodiff implementations which aren't easy to get right, I wrote a small toy version of a sparse rules-based autodiff here: https://github.com/rdyro/SpAutoDiff.jl

Another example (a much more serious one) is https://github.com/microsoft/folx

[gdalle]

You might also be interested in Spadina for Enzyme. There are no Julia bindings yet but I’d be excited if someone made them! https://c.wsmoses.com/presentations/weuroad23.pdf