|
|
|
|
|
|
by jremmons
2966 days ago
|
|
|
I feel like this article does not substantiate its core claim, "clever coders can beat tech giants." The main evidence given by the authors is that because Google lost to a much smaller company, Fast.ai, on a toy problem (training a model to 93% accuracy on the CIFAR-10 dataset) that small-time AI researchers can beat the big guys at their own game. This feels like a straw man argument and I'm not convinced. That said, I do think smaller, independent AI research groups can be impactful even if they aren't going to beat Google in a head-to-head competition; they just have to find problems the tech giants aren't working on yet :). |
|
|
Training small-ish datasets quickly is definitely not a toy problem. Most folks I know using neural nets in practice are using datasets of ~100MB. Some people do have giant datasets, but it's not the norm. And being able to train them quickly and cheaply is great for running experiments, as well as making the field more accessible to people with fewer resources.
If you're only interested in training larger datasets, this competition showed that the fastest Imagenet training on a single machine, and the fastest on publicly available infrastructure, also was done by fast.ai. The only better results were on TPU Pods, which are TPU machine clusters that are not publicly available. (It wouldn't be terribly hard to show similar results with a cluster of GPU machines, although it wasn't something that we have spent time on ourselves as yet.)
I strongly disagree with your claim that small groups can only compete with tech giants on stuff that tech giants aren't working on. There is a large amount of empirical evidence both now and throughout history that this isn't correct, and you have shown nothing to substantiate this claim. (I heard similar claims when Google first appeared: "a small group of Stanford researchers can't beat Yahoo.")