[eru]

Human can mostly only do these kinds of guesses for traveling salesmen problems embedded in 2d Euclidean space. But we have pretty good heuristics for these cases to kickstart a solver, too. Give a human a general graph with arbitrary edge weights, and they'll be dumbfounded.

(I don't think you even have to go all the way to an arbitrary graph, I suspect a decent sized graph with edge lengths embedded in 3d euclidean space will already confuse humans. Definitely once you get to 4d.)

[spywaregorilla]

My point is not that we should mimic humans. My point is that there's probably learnable but inexplicable heuristics you could learn for generally solving gradient descent problems just by the formulations on their own that a neural net would be good at.