[sythe2o0]

Better isn't always straightforward, sometimes unintuitive steps need to be taken to reach a better fitness value. The advantage of evolutionary methods over backprop is that evolutionary methods can take steps "backwards" and avoid local minima.

[chillee]

But local minima aren't a problem for modern neural networks that are optimizing in very high dimensional space.

[Houshalter]

It definitely is a problem. For instance there was a recent post about trying to teach a robot to put a board with a hole in it into a peg. It would just learn to shove the board next to the peg.

[chillee]

Sorry, saw this late.

It's a shown/kind of proven result that deep neural networks don't fall into local minima in the very high dimensional parameter space.

GANs and reinforcement learning are different. Research on getting those to converge to good local minima is still much more in its infancy. I don't particularly consider those just a "neural network", but sorry, I should have been more clear.

[Houshalter]

This thread is about reinforcement learning which definitely suffers from local minimas.

But even vanilla supervised nets suffer from local minima. Anyone who's played with them has encountered it. Here you can mess around with a neural net live in the browser and it very easily gets stuck if you try more than 3 layers (especially try the spiral dataset): http://playground.tensorflow.org/

[chillee]

That's why I said high dimensional neural networks. There's been a lot of literature explaining why local minima aren't a problem in very high dimension loss surfaces.

Check any of the literature on this subject: https://arxiv.org/abs/1611.06310v2

https://arxiv.org/abs/1406.2572

Local minima are something that people thought was gonna be a problem, especially back in the 2000s. They played around with small neural nets on toy examples such as yours, and thought it was intractable. It's the entire reason why neural nets fell out of the fashion in the early 2000s, and people moved towards techniques like SVM.

These toy examples don't generalize to high dimensions, and if you take a look at the literature, you'll see that the consensus agrees with my statement.

[Houshalter]

Ehh these theoretical results have questionable application to real life. Sure it might be very easy to learn simple correlations like "this patch of pixels correlates highly with the output '8'". But it's trivial to construct examples where neural nets get stuck in local minimas. For instance, try training a net to multiply two binary numbers.

Maybe with a billion neurons, just by random chance some of them would correspond to the correct algorithm and get reinforced by backprop. But very few NNs have layers larger than a thousand neurons. Because the cost of layers that big grows quadratically. And the chance of random weights finding the solution decreases exponentially.

One of the biggest reasons things like stochastic gradient descent, and dropout are used is because they break local minimas.