[Homunculiheaded]

My experience has been that none of the major Deep Learning libraries (Theano, Torch7, Caffe) offer support for OpenCL, whereas they all make it trivially easy to get models running on a CUDA GPU. On top of that NVIDIA has a library of deep neural network primatives[0], and I don't believe AMD offers anything similiar.

The general consensus I've seen is to just get an NVIDIA card if you're serious about working with deep neural nets on the GPU.

One thing that did surprise me was that there was no mention of using EC2 GPU spot instances for getting your feet wet. If you don't have access to a GPU with CUDA support you can get a spot instance for about $0.07 an hour to at least test out that you have your GPU code configured correctly (and you will see some performance gains). There are even a couple of AMIs out there with Torch7 and Theano already installed.

0.https://developer.nvidia.com/cuDNN

[timdettmers]

Thanks for elaborating, these are exactly the points why AMD's GPUs are just not used in deep learning.

AWS is great if you want to use a single or two separate GPUs. However, you cannot use them for multi-GPU computation as the virtualization cripples the PCIe bandwidth; there are rather complicated hacks that improve the bandwidth, but it is still bad. Everything beyond two GPUs will not work on AWS because their interconnect is way to slow.