[YeGoblynQueenne]

I honestly don't understand all this flowing uphill and flowing downhill talk. We advance science when we understand stuff. Untill we understand stuff, we don't have science, we just have stuff. Experimentation can come before or after, but science is the knowledge that comes with understanding that explains observations- not the experiments that generate observations.

People could still flow boats before Navier-Stokes? Yes, so people had boats, i.e. stuff. Now we have Navier-Stokes which is science, not stuff.

Btw, Yan LeCun knows this much better than me, but neural networks are already ancient. The first "artificial neuron", the Pitts & McCulloch neuron, was described in 1938. Frank Rosenblatt created his Perceptron in 1958. Kunihiko Fukishima described the Neocognitron, daddy of the Convolutional Neural Network, in 1979. Hochreiter and Schmidhuber described Long-Short-Term Memory Networks in 1995. Yan LeCunn himself used CNNs to learn to recognise handwritten digits in zip codes in 1989.

That's at least 30 years of research on deep neural nets- almost a human generation. Many of today's postgraduate students studying deep neural nets weren't even born when all this was being done. If this is just the experimentation phase before we pass on to the theorising and understanding phase- when are we going to get to the understanding phase? In 100 years?