[fspeech]

For one GPU speedup over CPU isn’t that dramatic for small to medium sized problems, e.g. MNIST or CIFAR that one would try algorithm ideas on. So I think it’s a stretch to see GPU as essential to the new algorithms. On the other hand for large problems like the original Alpha Go you need to figure out the distributed computing to really scale.

This isn’t to say that GPUs aren’t nice. They do save time or for the same amount of time let you produce more polished results, which means in a competitive environment everyone would use them.

[codesushi42]

Exactly. GPUs are necessary now, but did not originally herald the deep learning revolution.