[srean]

You are right. Unfortunately, many (doubly unfortunately, even in academia, well, many who switched careers in optimization to ML) think that machine learning is just optimization.

Regarding deep NNs, one should be careful with what one wishes for, because sometimes they come true. Landing up with the global optimum of that thing would likely be the last thing one wants.

The key to deep NNs is to do such a pathetic job of optimizing the loss that the generalization is good. A problem is that there several different ways of doing a job poorly, not all of them would generalize well. When I have my engineer hat on, I would rather not have lots of indeterminism on my watch if I can afford it. Too dang hard to maintain correctness of.

On the other hand if one has a "with high probability" style result where the probabilities are high enough to be practically relevant, then we have something more workable.

[_raoulcousins]

I don't understand why you don't want a global optimum. Is this obvious? Does the following paragraph explain it, because I don't see the connection.