[dhaivatpandya]

Interesting article. It is also possible to resolve constraints probabilistically (i.e. without gradient descent), which is a good approach for functions which are very "unsmooth." I wonder if it is possible to replicate this article using a probabilistic solver (e.g. http://poincare.github.io/DCFL/ - a Haskell probabilistic constraint solving lib. written by me).

[gh02t]

I think it'd also be cool to look at more sophisticated deterministic methods with these nice visualizations. I imagine a similar diagram that shows the expansion and contraction of the trust region in an L-M solver would be pretty enlightening as well. Or look at the path taken by steepest descent for pathological cost functions like the Rosenbrock banana.

I've done visualizations like this in Mathematica, which is really awesome for doing interactive visualizations, but it'd be fun to redo these in Haskell. I haven't done much plotting in Haskell, what libraries do people favor?