[fmvab]

I think this is unfair. Tens (hundreds?) of billions of dollars have gone into deep learning and, as I understand it, most effort consists of scaling up this golden goose and presenting its "new" successes in the best light possible. Every time throwing scale at the problem produces an impressive result, billions more dollars flow in, thousands of more people are nudged into the field. We got the first orders of magnitude of scaling "free" from GPUs, but the rest are going to come at considerable cost.

Whether or not there is a better alternative, if deep learning is in fact as over-hyped as the author claims, this could be a tremendous waste of money and intellect that could be spent on literally anything else, not just machine learning (maybe they could put those resources into crypto instead /s). That alone is enough reason to want intellectually honest skeptical takes, whether or not the author has a better idea. In addition, within AI, it makes it much harder for people for people to do anything else.

If there is a contraction in the field it will likely cause another giant AI winter. Somebody should start thinking of a use for all that compute.

The internal combustion engine did not require as much of a drain on resources before it produced results. Is deep learning actually making significant amounts of money, funding and valuations excluded?

[sdenton4]

What you're missing is that the gigantic initial success models have consistently gotten much much lighter after the hype cycle moves on from the initial release. In my own domain, WaveNet - speech synthesis vastly outperforming previous methods - went from fantastically expensive to being able to run on cheap phones. You can run bird song id on your phone in real time in the Merlin app. There's a proliferation of light neutral networks for lots of specific use cases, and it's already happening all around you.

[fmvab]

Thank you — that is definitely encouraging. It doesn’t sound like a killer app yet but it certainly seems to establish deep learning’s place at least as part of a modern software stack.