[nabla9]

The AI winter happens when outside hype and investment money overshoots what can be delivered in the timescale needed for good ROI. Disappointments are followed with periods of underinvestment.

Neural Networks are already decades old invention. Underlying all this new boom is the same basic architecture, alternating layers of affine transformations and nonlinear transformations trained with backpropagation and gradient descent.

What made the field suddenly explode was GPGPU's and bunch of tweaks that helped to solve vanishing/exploding gradients problems (started with RNN training layers and now with skip connections and handful of other techniques that make deep networks possible) combined with better regularization etc. I'm not trying to downplay the innovation, but from the larger point of view they are technical tweaks.

There is limit to where tweaking/scaling the current techniques can go. There needs to be leaps. Geoff Hinton made the case better in his "What is wrong with convolutional neural nets?" https://www.youtube.com/watch?v=Jv1VDdI4vy4 see also https://arxiv.org/abs/1711.11561 from Jason Jo, Yoshua Bengio. The same "what's wrong with" applies to the field in general.

Hinton's capsule networks are attempt to take another leap forward. Just like with his earlier work from 80's that culminated around 2006, it may take years and years of slow work to get there. Hinton suggests that there needs to be unsupervised learning revolution that comes up with something else than backprobagation. https://www.axios.com/artificial-intelligence-pioneer-says-w...