[orbifold]

As far as I'm aware there is considerable prior art, since solving this problem has been relevant since at least the 70s. Granted the barrier to entry was far higher then, then now, but we are talking about scientific / engineering applications and there the prior investment in specialised tools has always been justified. A contemporary example is http://www.dolfin-adjoint.org/en/latest/, but there are others. Sundials for example also both supports completely custom linear and non-linear solvers in conjunction with adjoint sensitivity analysis.

[adgjlsfhk1]

there is lots of prior art but it's still not ideal. the one you posted, for example only has reverse mode AD which sometimes is good, but forward mode is sometimes asymptotically faster (and has lower overhead, and some numerical stability issues)

[ChrisRackauckas]

Sundials does have forward sensitivities, though the implementation does not SIMD the calculations of the primal with the dual parts like direct AD of the solver does. This makes it a bit worse on forward mode than direct differentiation of a solver (with an optimizing compiler) would do, which was something that took us a few years to really pin down (we had to implement our own to really benchmark it, and got the same phenomena demonstrated in https://arxiv.org/abs/1812.01892). The nice thing about the direct forward mode is that it is easier to opt the extra equations out of the linear solver, something that AD systems won't do by default (but it can be done)