[_fh5n]

I understand that most people working with deep learning wouldn't want this type of thinking to spread amongst the public, and I surely don't want it either. But you have to be totally unaware of reality to think that DL is the definitive tool for AI. Most impressive results in DL in the past 2 years happended like this:

>deepmind steals people from the top ML research teams in univerisites around the world

>these people are given an incredible amount of money to solve an incredibly complex task

>a 6000 layers deep network is run for 6 months on a GPU cluster the size of Texas

>Google drops in their marketing team

>media says Google solved the AI problem

>repeat every 6 months to keep the company hot and keep the people flow constant

>get accepted at every conference on earth because you're deepmind (seriously, have you seen the crap that they get to present at NIPS and ICML? The ddqn paper is literally a single line modification to another paper's algorithm, while we plebeians have to struggle like hell to get the originality points)

I'll be impressed when they solve Pacman on a Raspberry Pi, otherwise they are simply grownups playing with very expensive toys.

Deep learning is cool, I truly believe that, and I love working with neural networks, but anyone with a base knowledge of ML knows better than to praise it as the saviour of AI research.

Rant over, I'm gonna go check how my autoencoder is learning now ;)

[dkarapetyan]

> I'll be impressed when they solve Pacman on a Raspberry Pi

I think this is the thing people don't quite get when they buy into the hype. These systems are extremely inefficient. Requiring terrabytes if not petabytes of data and basically a powerplant next to a data center to power the whole thing.

The work is valuable and pushing the boundary on what the hardware can do is great but so far all these things lack any kind of explanatory power and suck up a lot of energy to power the black boxes. DARPA recently put out a research program for making systems more efficient and adding explanatory capabilities to them (http://www.darpa.mil/program/explainable-artificial-intellig...). Ultimately that is the direction these things must be headed if they are to provide real value for the masses. Relying on a clever black box only takes you so far and is not beneficial in the long run because as these systems become more integrated into the institutions that drive large scale decision making they'll need to be held accountable for those decisions.

[DocSavage]

It's not all hype. We are using deep learning to solve real issues in our research, things that might not get done as well due to the representational learning aspect and our relatively small manpower. It's a definite advance when it comes to improving state-of-the-art and will have impact across all sorts of things, particularly small groups who have problems but don't have hundreds of people and ten years to figure it out.

The powerplant criticism is more true of the training phase and much less true of using the resulting networks on a larger scale. The good thing about the "hype" is that more resources are being directed into the field so they'll be more efficient processing platforms (e.g., ASIC or FPGA) and better delineation of what's really needed and if there are possible shortcuts (e.g., ReLU).

The black box problem will prevent its use in some systems, but even in some areas of medicine, it will be fine because medical AI must be used in conjunction with the final decision-maker, much like how Watson is being positioned. A deep learning system that detects anomalies in patient imaging with very high precision will be useful even if it can't explain why it thought it was an anomaly. It's quality control for the radiologist, etc.

[dkarapetyan]

I think that's a good use case. Such use cases are great and I agree it's not all hype but when everyone and their grandma has an "AI" startup you have to wonder what exactly "AI" means at that point.

[dwaltrip]

I can't speak of their other work, but as someone who plays go and follows go bots, the AlphaGo project was a phenomenal piece of work that shouldn't be discounted. Go is a very difficult game that requires great intuition and a deep understanding of the patterns.

Sure, maybe their timing was fortunate and it might have happened a year later anyways. But this is definitely not a guarantee, and they were still the ones to do it.

P.S. For anyone who is interested, another very strong bot is out in the wild now. It has been playing on the KGS go server the past week under the name Zen19L, and has a rank of 9d. Some great games have resulted as people challenge it.

[jfv]

So is image recognition, but I think the main contention is that solving Go required little additional cleverness to what has already existed. The ideas of self-play, Monte Carto methods, and neural networks are not new and not novel for the problem. Not that it's so trivial -- I'm sure it took a while to work out the exact architecture and details, but what of that work actually teaches you something?

[dwaltrip]

The combination of the methods was certainly novel as far as I am aware. Also, it was not clear ahead of time that it would actually succeed in defeating the best humans. Demonstrating that it could be done is a grear achievement.

There is also the symbolic value. It was a "coming of age" event to a certain degree. I believe go was the last classic board game that researchers had been longing to conquer.

I haven't looked deeply into this, but my understanding is that image recognition is still somewhat subpart outside of well-curated datasets. Is that not the case anymore?

[randcraw]

By that token, IBM's wins at chess and Jeopardy also deserve to be seen as "coming of age" events. And while the wins certainly showcased IBM's engineering prowess, I'm unconvinced Deep Blue or Watson moved the AI needle in a meaningful way. If they had, others would have followed in their technical footsteps. But AFAIK, no one has. I believe this lack of high tech repercussion will be true of AlphaGo too. Novel AI for game play just doesn't transfer outside the game.

[dwaltrip]

You don't think Deep Blue was inspirational to folks who were perhaps deciding if they wanted to pursue AI research?

The same holds for AlphaGo and Watson. Also, my understanding is that they are more technically interesting than Deep Blue. Given how recent these projects are, their legacy is only beginning to unfold.

I agree with you that applying this tech to domains other than games is no easy challenge. But I would be very surprised, in the long run, if events like this are not documented as key steps along the way at some point in the future.

[jfv]

My problem is that what people seem to be saying now is that "computers are smart enough to solve Go now, thanks to Google".

1) Too much credit is given to Google for this result. What I see as already a huge brain drain on society is only going to get bigger.

2) People are going to expect that if "computers are smart enough to play Go", they're smart enough to do ____. What goes in the blank? Very little right now, but I guarantee you investors and the public have a lot of ideas and think it's around the corner.

Hopefully a lot of good things will come out of this wave of AI, but who knows what or when. I think the point is that there may be a panic and contraction before anything really awesome happens, and a lot of that is going to be because of "stunts" (for lack of a better word) like this being overhyped.

[madgar]

> I'll be impressed when they solve Pacman on a Raspberry Pi, otherwise they are simply grownups playing with very expensive toys.

I'm pretty sure "grownups playing with very expensive toys" accurately characterizes >100,000 software employees in the US right now.

[dquail]

Agree generally. Except being unimpressed unless performance is achieved on sub Google scale hardware. Today's Google supermachine is tomorrow's raspberry pie. No need to artificially constrain our bounds. There is, after all, the inevitability of Moores law.

[dkarapetyan]

Moore's law has been dead for a while now. Most of the chip in your phone is powered off because otherwise it would burn up. Highly recommend watching this video: https://www.youtube.com/watch?v=_9mzmvhwMqw

[tekni5]

Very fascinating talk, and easy to understand.

Have there been any solutions proposed, in particular to the limitations of adding extra cores.

[socmag]

Really thanks for posting this video. Very cool to see Sophie Wilson talking so elequently and with obvious authority on a subject I and many others deeply care about. Great!

[DennisP]

True, but even without Moore's Law, using neuromorphic chips instead of general-purpose CPU/GPUs would likely be much more efficient. In the meantime it makes sense to use large server farms to emulate candidate neuromorphic architectures.

[dquail]

Interesting. I've yet to hear a Moores law is dead argument, so perhaps I should watch the video before commenting further. But the fact that most of the chip is turned off, doesn't falsify the fact that most of it still exists. Cooling it properly is a separate problem independent of computation no?

[tekni5]

The talk mentions that there is a physical law to how many cores you can add to a CPU before it becomes useless, even with parallel computing.

[typon]

> Today's Google supermachine is tomorrow's raspberry pie

No it isnt.

I really hope people stop spreading this myth.

[tim333]

I'm not sure solving Pacman on a Raspberry Pi would be terribly worthwhile. Deepmind's research into getting their software to play Atari games was interesting because they used a general algorithm that just received the values of the screen pixels and game score and learned from there. They used the same software for each game - it wasn't tuned or set up differently and so was a form of general AI if a very sub human one. It was interesting research that it worked as well as it did and played some games very well but wasn't good as Pacman because it lacked the ability to plan ahead.

They are primarily a research company, not a marketing setup. The latest stuff the released on letting the networks do something like dreaming which reduced the time to learn by up to 10x seemed interesting and I look forward to seeing how they do with Starcraft and the hippocampus. A lot of this stuff is cool because it gives insights into the human mind and how the brain works rather than practical gadgets. https://www.extremetech.com/extreme/240163-googles-deepmind-...