[JoeAltmaier]

Got to believe, this is like heroin. Its a win until it isn't. Then where will AI researchers be? No progress for 20 (50?) years because the temptation to not understand but to just build performant engineering solutions, was so strong.

In fact, is the researcher supposed to be building the most performant solution? This article seems alarmingly misinformed. To understand 'artificial intelligence' isn't a race to VC money.

[otoburb]

>>This article seems alarmingly misinformed.

I hate appeals to authority as much as anybody else on HN, but I'm not sure that we could say Rich Sutton[1] is "misinformed". He's an established expert in the field, and if we discount his academic credentials then at least consider he's understandably biased towards this line of thinking as one of the early pioneers of reinforcement learning techniques[2] and currently a research scientist at DeepMind leading their office in Alberta, Canada.

[1] https://en.wikipedia.org/wiki/Richard_S._Sutton

[2] http://incompleteideas.net/papers/sutton-88-with-erratum.pdf

[JoeAltmaier]

He's writing that article for a reason, to be sure. Its just not the one that the article says its about, I'm thinking.

[visarga]

AI as a field relied mostly on 'understanding' based approaches for 50 years without much success. These approaches were too brittle and ungrounded. Why return to something that doesn't work?

DNNs today can generate images that are hard to distinguish from real photos, super natural voices and surprisingly good text. They can beat us at all board games and most video games. They can write music and poetry better than the average human. Probably also drive better than an average human. Why worry about 'no progress for 50 years' at this point?

[JoeAltmaier]

Because, they can't invent a new game. Unless of course they were only designed to invent games, and by trial and error and statistical correlation to existing games, thus producing a generic thing that relates to everything but invents nothing.

I'm not an idiot. I understand that we won't have general purpose thinking machines any time soon. But to give up entirely looking into that kind of thing, seems to me to be a mistake. To rebrand the entire field as calculating results to given problems and behaviors using existing mathematical tools, seems to do a disservice to the entire concept and future of artificial intelligence.

Imagine if the field of mathematics were stumped for a while, so investigators decided to just add up things faster and faster, and call that Mathematics.

[visarga]

What GPT-3 and other models lack is embodiment. There are of course RL agents embodied in simulated environments, like games and robot sims, but this pales in comparison to our access to nature and the human society. When we will be able to give them a body they will naturally rediscover play and games.

Human superiority doesn't come just from the brain, it comes from the environment this brain has access to - other humans, culture, tools, nature, and the bodily affordances (hands, feet, eyes, ability to assimilate organic food...). AI needs a body and an environment to evolve in.

[YeGoblynQueenne]

>> AI as a field relied mostly on 'understanding' based approaches for 50 years without much success. These approaches were too brittle and ungrounded. Why return to something that doesn't work?

To begin with, because they do work and much better than the new approaches in a range of domains. For example, classical planners, automated theorem provers and SAT solvers are still state-of-the-art for their respective problem domains. Statistical techniques can not do any of those things very well, if at all.

Further, because the newer techniques have proven to also be brittle in their own way. Older techniques were "brittle in the sense that they didn't deal with uncertainty very well. Modern techniques are "brittle" because they are incapable of extrapolating from their training data. For example see the "elephant in the room" paper [1] or anything about adversarial examples regarding the brittleness of computer vision (probably the biggest success in modern statistical machine learning).

Finally, AI as a field did not rely on "understanding based approaches for 50 years"; there is no formal definition of "understanding" in the context of AI. A large part of Good, Old-Fashioned AI studied reasoning, which is to say, inference over rules expressed in a logic language, e.g. this was the approach exemplified by expert systems. Another large avenue of research was that on knowledge representation. And of course, machine learning itself was part of the field from its very early days, having been named by Arthur Samuel in 1959. Neural networks themselves are positively ancient: the "artifical neuron" was first described in 1938, by Pitts & McCulloch, many years before "artificial intelligence" was even coined by John McCarthy (and at the time it was a propositional-logic based circuit and nothing to do with gradient optimisation).

In general, all those obsolete dinosaurs of GOFAI could do things that modern systems cannot - for instance, deep neural nets are unrivalled classifiers but cannot do reasoning. Conversely, logic-based AI of the '70s and '80s excelled in formal reasoning. It seems that we have "progressed" by throwing out all the progress of earlier times.

____________

[1] https://arxiv.org/abs/1808.03305

P.S. Image, speech and text generation are cute, but a very poor measure for the progress of the field. There are not even good metrics for them so even saying that deep neural nets can "generate surprisingly good text" doesn't really say anything. What is "surprisingly good text"? Surprising, for whom? Good, according to what? etc. GOFAI folk were often accused of wastig time with "toy" problems, but what exactly is text generation if not a "toy problem" and a total waste of time?