[jpfed]

This made me think about code. I've long thought of incremental refactoring as "following the gradient" of good design principles, but this essay opened up new connections for me.

For example, I've heard elsewhere that great programmers possess the skill of being able to decompose large changes into smaller commits, making the code no worse as they go. Having the ability to see the small changes that get you where you want to go corresponds to finding the direction of change that gets you past the saddle point.

Random restart is a standard tool in the kit of stochastic gradient descent. But if local minima are rare, it may be overused. You don't have to rewrite your code if you can see the refactors and gradual improvements that get you where you need to go.