Hacker News new | ask | show | jobs
by adgjlsfhk1 1452 days ago
while there are c++ solvers, they are missing the flexibility needed to solve many types of problems efficiently. they almost all use finite differencing, which is inaccurate and slow, and they rarely let users customize the linear solve and nonlinear solve algorithms. What the Julia diffeq suite is working towards is complete control over solver options, while picking good defaults when not customized. They let you pick forward and backward mode AD (or a combination of the two), problem specific preconditioning methods, custom root finding algorithms (including gpu based linear solvers), arbitrary precision methods for verification (or if you're just paranoid), interval arithmetic, linear uncertainty propegation, and more.
1 comments

As far as I'm aware there is considerable prior art, since solving this problem has been relevant since at least the 70s. Granted the barrier to entry was far higher then, then now, but we are talking about scientific / engineering applications and there the prior investment in specialised tools has always been justified. A contemporary example is http://www.dolfin-adjoint.org/en/latest/, but there are others. Sundials for example also both supports completely custom linear and non-linear solvers in conjunction with adjoint sensitivity analysis.
there is lots of prior art but it's still not ideal. the one you posted, for example only has reverse mode AD which sometimes is good, but forward mode is sometimes asymptotically faster (and has lower overhead, and some numerical stability issues)
Sundials does have forward sensitivities, though the implementation does not SIMD the calculations of the primal with the dual parts like direct AD of the solver does. This makes it a bit worse on forward mode than direct differentiation of a solver (with an optimizing compiler) would do, which was something that took us a few years to really pin down (we had to implement our own to really benchmark it, and got the same phenomena demonstrated in https://arxiv.org/abs/1812.01892). The nice thing about the direct forward mode is that it is easier to opt the extra equations out of the linear solver, something that AD systems won't do by default (but it can be done)