[adgjlsfhk1]

> g(x, i) = sin(ix) / i

That's an example where the derivative does not exist.

> I guess another issue that could happen is that the original sequence converges, and the derivative sequence converges, but they converge at different rates.

This is a lot closer to what's happening in the video. For a potentially simpler example than an ODE solver, if you had a series evaluator that given a series evaluated it at a point, AD would need a similar fix to make sure the convergence test is including the convergence of the derivative.