[nora4]

Deep Learning is currently an empirical science guided by intuition of practitioners. A main principle in experimental sciences is that a theory without predictive power is not considered a full-fledged theory. As such, unless they are interesting predictions coming from their theory (rather than only barely justifying existing empirically observed phenomena), this is just speculative theory that I would not use the phrase "Foundations Built" for.

As an example of this general litmus test for a theory see e.g. Eddington's confirmation of GR: https://en.wikipedia.org/wiki/Tests_of_general_relativity#De... . If there are hitherto unknown phenomena in DL predicted by this theory then I'd stand corrected and concede that there may be something to these theories.

[fizx]

Read this as "some foundations built". Even that is much-needed progress.

[hyperpallium]

Yeah, it's not foundations like elementary or axiomatic.

An actual theory of ANN and then of NNN will be of more momentous import than of any previous phenomena, will usher in a fundamentally transformative age where we understand ourselves, and will surely require a fundamental breakthrough in pure mathematics... probably a great many.

[matt4077]

The article seems pretty upfront about the fact that we are nowhere near a solid theory. One of many quotes to that effect: “ […] beginning to build the rudiments of a theory of neural networks.”

The main point seems to be why such a theory is highly desirable. To that end, it does a far better job than the somehat tired Feynman aphorism about good theories “giving us more than we give them”. Namely that the current state of the art of “intuition” and vast trial-and-error runs being somewhat embarrassing for a field so closely related to math and statistics.