[GuiA]

Why do we need to explicitly design architectures such as the "imagination encoder" the article describes? A proposed long term goal of deep learning is to have AI that surpasses human cognition (e.g. DeepMind's About page touts that they are "developing programs that can learn to solve any complex problem without needing to be taught how"), which was not explicitly designed in terms of architectural components such as an "imagination encoder".

Shouldn't imagination and planning be observed spontaneously as emergent properties of a sufficiently complex neural network? Conversely, if we have to explicitly account for these properties and come up with specific designs to emulate them, how do we know that we are on the right track to beyond human levels of cognition, and not just building "one-trick networks"?

[habitue]

The human brain has specialized structures in it, it isn't a homogeneous mass from which all parts of human cognition emerge once you have enough brain cells (see elephant brain size vs. human brain size). If you've ever seen anything else designed by evolution, you'll know it generally tends to be a grab-bag of weird tricks all combined together in a way that somehow works. We don't know what all the tricks are, nor which are necessary or sufficient to create human-like intelligence.

There are also a lot of indications that ultimately you need some tricks (i.e. specialized portions of the architecture that bias the kinds of solutions the AI can learn) to be able to learn effectively in the environments we're interested in. For example, we know that there is a time dimension to agent tasks, and that objects don't pop in and out of existence, they tend to exist continuously. These are biases we are free to add to a learning system without worrying about it limiting the ultimate intelligence of the system.

In the limit, the No Free Lunch theorems indicate that there's no such thing as a general learning system that doesn't sacrifice performance on some kinds of tasks. The goal of AI research is to sacrifice performance on tasks that we'll never encounter in favor of getting good performance on tasks we care about.

[GuiA]

> If you've ever seen anything else designed by evolution, you'll know it generally tends to be a grab-bag of weird tricks all combined together in a way that somehow works. We don't know what all the tricks are, nor which are necessary or sufficient to create human-like intelligence.

That is precisely the core of my interrogation. The papers mentioned in the article seem to be about "hand designing" the weird tricks; shouldn't the goal be to build a system that enables the emergence of these weird tricks without involving human design?

[PeterisP]

> shouldn't the goal be to build a system that enables the emergence of these weird tricks without involving human design

It depends on your goals - if your goal is to build a system that can perform smart actions (e.g. build/simulate something comparable to a brain), then that's not required (it may happen to be useful, or not); if your goal is to build a system that can create and build systems that can perform smart actions (e.g. build/simulate something comparable to the evolution process of an intelligent species) then it should.

[habitue]

> shouldn't the goal be to build a system that enables the emergence of these weird tricks without involving human design?

Two comments:

1. Just because evolution came up with them for humans, doesn't mean if we run an evolutionary algorithm we'll come up with an intelligent system in any reasonable amount of time. There's no reason to believe it's easy to evolve such systems given that we only know of one human-level intelligence in the universe, and it seems to have taken billions of years to come about.

2. This is unnecessarily tying our hands. Evolution often builds very inefficient, overly complicated versions of things that can be simplified dramatically once humans understand the underlying principles behind why they work. In addition we have a huge body of theoretical work on planning, decision theory etc that improves dramatically on our natural learning processes that we can take advantage of. We get no points for not "cheating" here.

[eduren]

Hand designing is the only feasible option available to us. A system that could architect itself would either need:

A bootstrap intelligence in order to self-plan, self-experiment, and self-modify. Escher hands drawing each other basically... or

Similar conditions to our only known spontaneous intelligence (us). That includes some sort of base code (genetics), competitive environments for rewarding good architectures, and lots of time in simulation. No guarantee this would work either.

[eli_gottlieb]

>Shouldn't imagination and planning be observed spontaneously as emergent properties of a sufficiently complex neural network?

No! There's never been any scientific guarantee that "sufficiently complex" neural networks will give rise to anything in specific as an "emergent property", let alone human cognitive abilities like imagination and planning.

>how do we know that we are on the right track to beyond human levels of cognition, and not just building "one-trick networks"?

Steps to write a deep learning paper (from the Cynic's Guide to Artificial Intelligence):

1) Use a training set orders of magnitude larger than a human could learn from, build a one-trick network that gets superhuman performance on its one trick of a task.

2) Hype it up.

3) Research funding and/or profit and/or world domination!

(World domination has never been supplied when requested.)

[eduren]

>Shouldn't imagination and planning be observed spontaneously as emergent properties of a sufficiently complex neural network?

Not necessarily. I think it comes down to what you mean by "sufficiently complex". If we took a classic feedforward Multi-Layer Perceptron and gave it massive amounts of good data, a long time to train, and a nearly unbounded network size, I'm not sure it would ever develop architecture within itself to plan or develop a robust internal model.

Our neurology took millions of years/generations to get where it is today though natural selection. We might want to tip the scales a bit by engineering the broad architectural pieces and letting emergent behavior fill the gaps.

Although it would be fun to try producing human level intelligence by seeding a physics simulation of primordial soup and letting it run for millions of "years", I don't think that's feasible for most researchers.

[virtuabhi]

> Although it would be fun to try producing human level intelligence by seeding a physics simulation of primordial soup and letting it run for millions of "years", I don't think that's feasible for most researchers.

And what would be the seed for random number generator?

[eduren]

"Let there be light"

[Gregaros]

>Shouldn't imagination and planning be observed spontaneously as emergent properties of a sufficiently complex neural network?

Why would you think that? We have no theoretical knowledge of how human "consciousness" emerges, and obviously no experimental data either.

>Conversely, if we have to explicitly account for these properties and come up with specific designs to emulate them, how do we know that we are on the right track to beyond human levels of cognition, and not just building "one-trick networks"?

We don't know how far the path leads, but the capabilities of the past 5-ish years progress are leaps and bounds beyond what anything else can do, and this is part of the work of pushing further down that path.

[tpeo]

Well, nobody is forced to create models they don't believe in.

But besides some models being useful, as that old adage says, some are also useful-er than others. Adding stuff such as "imagination" functions as a constraint on the number of behavioral patterns that we are willing to consider, and that might lead us to find that one which we're looking for (i.e. "intelligent behavior") faster than a naïve approach.

Besides, it might not be the case that the likelihood of observing "intelligent behavior" increases over the complexity of the behavior generating process.

[landon32]

Right now we create feed forward neural networks that have a very specific architecture so that we can easily train them. If we want a really "general purpose" sufficiently complex neural network we would need it to use way, way, way more computation power and take up many 100s of times as much data. It would be insane. So instead of having an amorphous blob of artificial neurons, it's much easier to design clever components and assemble them in a general purpose way.

[hacker_9]

Imagination doesn't seem learnt to me. Instead learning new concepts adds to the toolbox so to speak.