|
I am a mathematician by trade, and was doing development along with other stuff (reverse engineering and security work, first in my own company, then at Google). So ... 1) I think working knowledge of ML is extremely useful to many developers, and generally under-taught in universities. See the old Joel article which mentions "Google uses Bayesian filtering like MS uses the IF statement" http://www.joelonsoftware.com/items/2005/10/17.html). A well-rounded developer should know the basics (Logistic regression, SVMs, know some things about CDNNs etc.), it will make him much more adept at problem-solving. I suspect Google's internal push to get people up to speed is not to turn them all into ML researchers, but rather to make sure that everybody "knows the basics well enough". So I think it is useful to teach developers about the things ML has to offer. 2) Mathematically, it seems that in ML the "engineering" side has run far ahead of the theory side. The sudden breakthrough in the mid-2000s is IMO still not fully understood - and parts of it may have been very accidental. Initially, it was thought that pre-training was the big breakthrough, but it is quite unclear what the big breakthrough was. It could be that simply the increase of data / compute sizes and the switch to minibatch-SGD explains why modern DNNs generalize well (interesting paper on the topic: https://arxiv.org/abs/1509.01240). There is a lot of good mathematics to be written, but I am not sure whether the folks at Google will write it - given the incentive structures (performance reviews, impact statements) it is unlikely that somebody gets promoted for "cleaning up the theory". 3) From a development perspective: There are a ton of interesting engineering problems underneath the progress in ML. If you look at Jeff Dean, he is a superstar engineer, not necessarily a mathematician, and a lot of the progress the Google Brain team made were engineering advances to scale / distribute etc. - so by training the engineers in ML, you also get to have better infrastructure over time. So I don't think they are sending "developers into ML swamps"; I think they are trying to reach the point where "Google uses DNNs like MS uses IF". Cheers,
Thomas |
Google, Facebook, M$ Research, and perhaps Yahoo are extreme outliers. They have zottabytes of broad unstructured text data, so they mine it. Everybody else has megabytes of narrow structured data, most of it commercial transations of their products. That stuff has already been effectively mined by traditional basic OLAP methods. Most/all of the value has been extracted.
Mainstream software apps have yet to show the value of using ML. Such apps have access to very limited data of very narrow relevance. The utility of ML in such domains isn't new; it's classic optimization. Or it's bayesian anticipation. But it's not a game changer. Frankly, the use of ML in most mainstream apps is more likely to add distraction and annoyance as the computer mispredicts your intent -- like Microsoft Bob did.
Maybe "life in the cloud" will create new opportunities for smarter software. But I definitely don't want free apps making their own decisions when to notify me. I guarantee that will get old immediately. So how will this work? Frankly, I can't guess. Like Apple's iAds, programming ML into the mainstream or cloud sounds like an idea that will serve the software / cloud vendor far better than the user.