[arbre]

I don't believe in "everyone should work on machine learning". I worked on several deep learning models but I don't really like it. It is a very different job than software engineering in my opinion. ML is more about gathering data and tuning the models as opposed to building stuff. I have spent months working on models and barely wrote any code. It is more efficient to have ML experts focus on the modeling and software engineers use the model.

I do believe however that some experience is needed to understand what is possible and best benefit from existing tools or to be able to communicate with machine learning engineers about your needs.

[giardini]

I concur. ML isn't programming per se; it is experimental problem-solving with a particular dataset and algorithm. Your result may/not work well, may/not generalise, and will almost undoubtedly not contribute anything new to any discipline, even to ML. When all ML work is done we'll have great pattern recognizers but nothing remotely akin to thought. And we won't understand how they work or the best way to build the next one. It isn't AI, although it is a part of AI, just as the visual system is part of AI.

I was reading Domingos' "The Master Algorithm" several days ago and a mathematician inquired about the book. He knew a group of ML developers. His opinion was that "ML doesn't look very interesting: all you do is play with the parameters, turn the knobs, and/or change the model until something works. There's no real progress there; nothing substantial."

Rather than sending a batallion of bright developers into the ML swamp where they will largely be frustrated, learn little and contribute less, I'd be tempted to guide them into other fields.

[tdullien]

I am a mathematician by trade, and was doing development along with other stuff (reverse engineering and security work, first in my own company, then at Google). So ...

1) I think working knowledge of ML is extremely useful to many developers, and generally under-taught in universities. See the old Joel article which mentions "Google uses Bayesian filtering like MS uses the IF statement" http://www.joelonsoftware.com/items/2005/10/17.html). A well-rounded developer should know the basics (Logistic regression, SVMs, know some things about CDNNs etc.), it will make him much more adept at problem-solving. I suspect Google's internal push to get people up to speed is not to turn them all into ML researchers, but rather to make sure that everybody "knows the basics well enough".

So I think it is useful to teach developers about the things ML has to offer.

2) Mathematically, it seems that in ML the "engineering" side has run far ahead of the theory side. The sudden breakthrough in the mid-2000s is IMO still not fully understood - and parts of it may have been very accidental. Initially, it was thought that pre-training was the big breakthrough, but it is quite unclear what the big breakthrough was. It could be that simply the increase of data / compute sizes and the switch to minibatch-SGD explains why modern DNNs generalize well (interesting paper on the topic: https://arxiv.org/abs/1509.01240). There is a lot of good mathematics to be written, but I am not sure whether the folks at Google will write it - given the incentive structures (performance reviews, impact statements) it is unlikely that somebody gets promoted for "cleaning up the theory".

3) From a development perspective: There are a ton of interesting engineering problems underneath the progress in ML. If you look at Jeff Dean, he is a superstar engineer, not necessarily a mathematician, and a lot of the progress the Google Brain team made were engineering advances to scale / distribute etc. - so by training the engineers in ML, you also get to have better infrastructure over time.

So I don't think they are sending "developers into ML swamps"; I think they are trying to reach the point where "Google uses DNNs like MS uses IF".

Cheers, Thomas

[randcraw]

I don't think your points are invalid, but I think you overvalue the data that's available and relevant to most programming tasks. And without novel data, ML can offer little novel value.

Google, Facebook, M$ Research, and perhaps Yahoo are extreme outliers. They have zottabytes of broad unstructured text data, so they mine it. Everybody else has megabytes of narrow structured data, most of it commercial transations of their products. That stuff has already been effectively mined by traditional basic OLAP methods. Most/all of the value has been extracted.

Mainstream software apps have yet to show the value of using ML. Such apps have access to very limited data of very narrow relevance. The utility of ML in such domains isn't new; it's classic optimization. Or it's bayesian anticipation. But it's not a game changer. Frankly, the use of ML in most mainstream apps is more likely to add distraction and annoyance as the computer mispredicts your intent -- like Microsoft Bob did.

Maybe "life in the cloud" will create new opportunities for smarter software. But I definitely don't want free apps making their own decisions when to notify me. I guarantee that will get old immediately. So how will this work? Frankly, I can't guess. Like Apple's iAds, programming ML into the mainstream or cloud sounds like an idea that will serve the software / cloud vendor far better than the user.

[giardini]

I don't know why randcraw is being downvoted here: his/her points are vital clarifications.

Humans have been gathering and analyzing data for thousands of years. We have _not_ waited for Google's latest ML or neural nets to do analyses. Otherwise I'd be carving this post onto a stone for future generations to peruse.

The valuable and understandable AI, the step that will make a difference, isn't in "big data" - it's in figuring out how to do what those humans have been doing all those thousands of years.

[serge2k]

> I don't know why randcraw is being downvoted here

I can't speak for anyone else, but "M$"

[dgacmu]

Think outside consumer-facing applications. Medicine, biology, geology (oil, gas, and mining), finance, transportation. Tons of data, tons of dollars, and important problems.

[randcraw]

I work in a big pharma analyzing image and experimental data. In a prior life I analyzed social cliques from vast numbers of user transactions. In both cases it seems like greater volumes of data should lead to deeper insights. But as it happens, the amount of useful actionable information in that data was surprisingly limited.

Often the available sensors/assays failed to detect reliable info. Or the phenomenon of interest interdepended on too many variables expressed with too great a dynamic range for us to detect reliably or model usefully. (The present lull in genomics R&D illustrates this well, as do automated interpretation of signals like EEG and NMR spectra.) And the signals that we can extract are often uninterpretable or sporadic. Alas, gathering more data won't yield more signal. Given the present limit on sensor resolution, you just get more mixed signals.

The potential of all ML is limited by the depth of the data that are essential for the discrimination of subtler signals. In the domains you mention (medicine, biology, geology, other sciences) I'm convinced we need better sensors more than greater amounts of the same data available now. We need better hypotheses which lead to better ideas of where to look and what to look for. In general, ML can't help with that. Until we better imagine how the mechanism might work, our questions remain too vague.

To wit, I'm afraid that applying ML to most software apps will suffer from the same limited ROI. I suspect that most app and user data is too shallow for mining to add appreciable value, no matter how clever it is.

[giardini]

Most of that data isn't "big data". And most of it has been analyzed thoroughly. Sure, ML will be used to re-analyze it, but with mostly the same results. As randcraw states "Most/all of the value has been extracted."

Only a wild-eyed ML "gold digger" could imagine that there is a vein of gold in those mines. The reality is that, with few exceptions, we'll find more lumps of coal.

Perhaps I should switch from an ML swamp metaphor to an ML mine metaphor? <--Hah! Do that with ML!

[LeanderK]

I think it depends on the job. Maybe a web-developer has lesser gain from extensive knowledge in ml. But I agree that every computer scientist (whether he works as an software engineer or not) should have some knowledge of ml, there are many things in the curriculum that are not as important as ml.

As a snarky remark: Maybe i am not yet qualified enough for real criticism as an cs-student, but i don't like it such sharp destinations between engineering and theory. All the "trial an error" in ml can be a useful guide to solving the theory. Also i guess the work of Jeff Dean is quite often more theoretical as the work of an average engineer. While i feel that if we have not developed a theory behind such tools, we have not really understood them, no one knows how komplex these things really are. I think/feel this makes ml-related engineering harder than software projects with a well understood theory

I just hope there are enough computer-scientists/mathmaticians at universities (or google ;) ) sharply looking on all the progess made in ml from the engineering side and asking themselves "what does that really mean?", because thats a hell of an interesting problem.

I may be wrong, my lecture on ml is next semester ;)

[giardini]

Or perhaps Google's ML career path is largely a ruse, a Golgafrinchan Ark Fleet Ship B, that Google is using to trim a bloated developer pool?!8-))

http://hitchhikers.wikia.com/wiki/Golgafrincham

[visarga]

> And we won't understand how they work

Is this a critique of the human mind or a praise of AI?

> When all ML work is done we'll have great pattern recognizers but nothing remotely akin to thought

Maybe our brains too are nothing but pattern recognizers. Maybe they are nothing but chemical reactions, or energy fields. But being reductionist about AI won't help us understand it either.

[zby]

What we need is models that are more retrospectable - so that you can find the rules that it learned. Most of the time they will be too complex for any human understanding - but from time to time we'll find something interesting, something that we can build other things upon.

I have never used neural nets etc - but with simplified bayes spam filters this was possible and quite useful. I used to check which words were pushing a text into one or other category and which did not (when they should).

[ioeu]

What fields would you guide them to instead?

I may be one of the developers you speak of (with academic aspirations), presently considering my path forward.

I'm sceptical if going down the ML swamp is the best way forward.

[etangent]

I don't know. I know some engineers who have spent months going back-and-forth over communication protocols while barely writing any code, yet somehow their job is considered to be quite core to software engineering. I don't really see how fine-tuning communication protocols is fundamentally different from fine-tuning machine learning models. But overall, I agree with your sentiment: different things are different and appropriate for different people.

[serge2k]

Wouldn't that be more akin to the design of the models?

[personjerry]

Hi arbre, would you mind explaining what is possible and what benefits from existing tools in machine learning at the moment? I am clueless and find ML rather frustrating to get into.

[arbre]

I meant that learning about ML and getting some field experience helps you figuring out when to use ML and how. For how to get into, there are a lot of resources and state of the art algorithms/papers/implementations are freely available. For me working on ML projects at my job and talking to some experts was ideal, but I am sure it is possible to learn on one's own with enough motivation. Good luck!

[personjerry]

Ah, yes, I understood what you meant (and thank you for pointing to where I should look next!). I was hoping, too, that you might share your ML knowledge in layman's terms.

[dgacmu]

Absolutely. In general, ML needs a collaboration between ML expertise and application domain expertise. It's very helpful if there's someone who can help bridge those two - enough app experience to understand the domain deeply, and enough ML experience to know what questions to ask of the ML gurus and what pitfalls to expect. As I see it, that's one of the goals of the ML ninja program.

[zmj]

This is how software eats your job.

[srtjstjsj]

> "everyone should work on machine learning"

> software engineers use the model.

You aren't disagreeing.

[lunchTime42]

Sounds like a job machines could do..

[uola]

"Moving data around" is what a lot of software engineering is these days. Facebook, Google etc. are more data companies than software companies (and probably close to media than communcations companies).