[koute]

If you're going for GPU support then please do not use CUDA. It's non-portable (NVIDIA only), proprietary, clunky and it requires a huge SDK which is very awkward to install. I highly recommend OpenCL which is portable, requires no SDK, is open and for properly optimized code is just as fast as CUDA.

[nl]

While this sounds attractive, OpenCL performance just doesn't seem to be there at the moment.

In CNN benchmarks[1] the CL based implementations consistently finish at the bottom. I think this is mostly because of the weak nVidia OpenCL implementations, but that doesn't help, since most people use nVidia (eg, Amazon GPU instances are nVidia based)

[1] https://github.com/soumith/convnet-benchmarks

[ben-schaaf]

Another nice thing about OpenCL is that it is also supported by non-GPU hardware. Kronos maintains a pretty impressive list: https://www.khronos.org/conformance/adopters/conformant-prod...