Hacker News new | ask | show | jobs
by obviuosly 2642 days ago
Various researchers, e.g. Werbos, Linnainmaa, Bryson and Yu-Chi Ho have been doing backprop in neural networks before Hinton et al. Hinton was merely a popularizer of the idea.
2 comments

Wright brothers were most likely not the first to achieve heavier-than-air powered flight. But they made the first controlled, sustained flight. Controlled flight was the real breakthrough in aviation.

It's the same with Hinton et. all. They were among many other pioneers, but what really sets them apart is that they made it all work. They also analyzed why it works.

> In 1890 Clément Ader made the first manned, powered, heavier-than-air flight, of 50 m, in his bat-winged monoplane.

Seems disingenuous to not mention that this device only worked due to ground effects and "flew" 8 inches off the ground (according to Wikipedia).

> but what really sets them apart is that they made it all work [citation needed]

Werbos and others were very aware that the method worked and also of the implications of it:

> This approach makes it possible to develop generalized, adaptive artificial intelligence, capable of achieving results comparable to what is discussed in science fiction

http://werbos.com/Neural/SensitivityIFIPSeptember1981.pdf

The Hinton cartel happened to be in the right condition that allowed them to ignore the politically correct BS starting at the turn of the century which rendered the study of neurons, IQ and intelligence massively unpopular; and ruthless enough to only cite one another and call themselves the "fathers of deep learning" rather than actually citing the originators of these ideas.

Octave Chanute in the case of heavier than air flight. He was the spark and the hub that connected most of the pioneers.

https://en.wikipedia.org/wiki/Octave_Chanute

For reference:

https://en.m.wikipedia.org/wiki/Gustave_Whitehead

Ive seen some debate around Gustave Whitehead. There apparently is some evidence that he achieved powered flight as early as 1899.

It is claimed, he did not speak English very well. So it’s suspected he just didn’t publicize his work as much.

Replicas of his early flying machine designs have been shown to fly. Including one from 1901:

https://en.m.wikipedia.org/wiki/Whitehead_No._21

"merely"

People deserve credit for taking an idea and running with it. Without Hinton's efforts (in 1987, 2006, or 2012), deep learning would have never seen the explosive growth.

"would have never..." it's not possible to make such an assertion unless you have access to at least one copy of this universe that doesn't include him. Maybe in all the other universes the exact same thing happened except someone else did it. Maybe in some of those universes, without him and his popularizing of this technique, some other better technique flourished. Who knows?

Stating "would never have" is nothing more than telling a story.

> People deserve credit for taking an idea and running with it.

Yes, as popularizers.

> deep learning would have never seen the explosive growth

This is extremely unlikely to be true. I think the explosive growth occured because neurons and intelligence were unpopular in the 2000s because of PC, deferring progress in this area to occur explosively later. The Hinton cartel were not the only people who were aware that AI research "has become silly" (in his own words).

But is PR what Turing award is for though ?
PR is what awards are all about
PR is what all science is about. Without communicating ideas, science is not useful. The trio above managed to communicate deep learning ideas to the wider vision and NLP community such as by collaborating with them to come up with benchmarks which would prove usefulness of deep learning, but also to industry and general public. Schmidhuber did very little on these fronts, and even his technical contributions while laudable do not compare with the awardee's.