Hacker News new | ask | show | jobs
by yu3zhou4 942 days ago
Specifically for neural networks, is there any alternative for backpropagation and gradient descent which guarantee finding the global minimum?
1 comments

Unlikely given the dimensionality and complexity of the search space. Besides, we probably don’t even care about the global minimum: the loss we’re optimising is a proxy for what we really care about (performance on unseen data). Counter-example: a model that perfectly memorises the training data can be globally optimal (ignoring regularization), but is not very useful.