[arise]

There's also the AI adage that you don't get to the moon by building successively larger ladders... all this deep learning stuff is great and will unlock amazing value (esp if we fund open data+models), but we run the risk of exhausting consumer and government patience if we keep promising the world with this single family of techniques.

[obblekk]

That's interesting and something I hadn't heard before.

Any pointers to further reading on why deep learning might be a dead end?

[vba616]

How about a simple explanation of why it seems palpably absurd to some people that it will not be a dead end?

Consider a teratoma or lab-grown organ. Is grafting or engineering another attached sensory mechanism going to bring it closer to being an ordinary organism?

I think whether capabilities are super- or sub- human is a red herring.

Even a really primitive organism is still taking all of its computational capabilities and outputting, implicitly, decisions in one context that is its perceived reality.

A collection of computation and perception modules does not do this, without something else.

I don't think developing "something else" is obviously impossible or would require magic. But I'm not sure anyone sane would want to create it when it inherently creates unlimited risk of running amok. This is what the LessWrong people are afraid of, aren't they?

[danaris]

I think it's important to distinguish between something that's a "dead end" in that it will take you nowhere interesting, and something that's a "dead end" in that it will, after a certain point, cease advancing.

The former would imply that there is no point using deep learning and other similar techniques at all, and is the common implication when people say something is a "dead end".

The latter is what I believe the current generation of machine learning to be: I do not believe it will lead to AGI, and I am skeptical that it can do a great deal more than what it has already done (it can continue to refine the types of things it already does, and I expect it to do so, but I don't think it will open up new categories of things it can do many more times). But despite that, it does do some very cool things now, and as they are refined, I think they can be commercially successful and generally beneficial tools.

[burnished]

They didn't say it was a dead end. I think their observation is rooted in gradient descent and how it improves but gets stuck in local maximas, when that solution might never be good enough. Hence the ladder. No matter how good you get at ladder making it with take you to the moon, but it will get you closer the entire time you improve at it.

[PeterisP]

For a contrasting viewpoint, there's always Sutton's Bitter Lesson - http://incompleteideas.net/IncIdeas/BitterLesson.html

[TremendousJudge]

Not sure if the analogy holds. Human brains work on the same principle as lizard brais, no change in strategy was needed to go from lizard intelligence to human intelligence. I'm not saying that DL holds this promise though.

[eternalban]

https://en.wikipedia.org/wiki/Triune_brain

There are distinct architectures and our brains have additional architectural features that lizards' lack.

https://en.wikipedia.org/wiki/Basal_ganglia

https://en.wikipedia.org/wiki/Limbic_system

https://en.wikipedia.org/wiki/Neocortex

[TremendousJudge]

Yes, but unlike a ladder and a rocket, they are not fundamentally different. This is why I don't think the analogy holds. Evolution iterated over the same model, never starting again from scratch.