[wladimir]

Everything can be done in C/C++ though, and while it'd be harder, ML is an area where the gain in speed

Don't forget machine learning generally involves a lot of experimentation, and this is easier with higher-level languages. Hand-optmization always makes assumptions about specific data structures and details which can be hard to change later.

You can get very close with code-generating high-level languages though. See Theano (http://deeplearning.net/software/theano/), for example. It uses a functional approach to compose the formulas for (c-)ANNs, and has advanced features such as automatic differentiation. Scalable generated GPU code can beat the pants of even the fastest hand-written C loops. Sure, hand-optimized GPU code can perform even faster but in my experience that is usually not worth the trouble.