[ogrisel]

The old backprop from the 80's / 90's is still in use and the primary way to train deep nets. We tend to call it SGD (on a composition of differentiable functions) nowadays but it's the same algorithm.