[rmelly]

(One challenge: though neural networks generalize very well, we still lack a decent theory to describe them, so much of the field proceeds by intuition. This is both cool and extremely bad. “It’s amazing to me that these very vague, intuitive arguments turned out to correspond to what is actually happening,” says Ilya Sutskever, research director at OpenAI., of the move to create ever-deeper neural network architectures. Work needs to be done here. “Theory often follows experiment in machine learning,” says Yoshua Bengio, one of the founders of the field. Modern AI researchers are like people trying to invent flying machines without the formulas of aerodynamics, says Yann Lecun, Facebook’s head of AI.)

One of the most interesting aspects of the field - we don't have robust ways of predicting what will work without trying it.

[arcanus]

> Modern AI researchers are like people trying to invent flying machines without the formulas of aerodynamics

An apt analogy, given that flying machines were designed by individuals without a robust understanding of fluid mechanics.