[Fronzie]

http://ceres-solver.org/ works well, in my experience. It does require code to be templated, which might increase computation time. It is possible to explicitly define derivatives for individual functions, this can help preventing numerical instabilities in corner cases.