[brutus1213]

I disagree partly. Cuda is defacto in DL at the moment, and is very well designed. It has academic roots and the neat ideas shine out.

Why did I say partly? So many optimization layers exist already (e.g. BLAS/cuBLAS). One may not really need to get down to the CUDA level.

[skohan]

What if anything does Cuda do better/differently than OpenCL?

[keldaris]

CUDA itself does only one thing better - it lets you write kernels in C++ rather than a fairly restrictive subset of C. Far more importantly, it has a large ecosystem of tooling (including excellent profiling tools), libraries, documentation and open source projects out there. Comparatively speaking, OpenCL is a barren landscape.

[codehd7]

SYCL layer/standard adds "C++ Single-source Heterogeneous Programming" for OpenCL: https://www.khronos.org/sycl/

[keldaris]

That it does and there's also plenty of convenience wrappers of various kinds for different languages. Unfortunately, none of that can address the ecosystem issue, which is the one that really matters.

[goldenkey]

Cuda has Cooperative Groups now on Volta and Turing architectures. This allows for synchronization between entire workgroups rather than just locally. So you can pretty much keep your entire job on the GPU even if it involves multiple kernels. Really important for complex jobs where performance is a must.

https://devblogs.nvidia.com/cooperative-groups