[wall_words]

The performance graph is deceptive for two reasons: (1) Leaf with CuDNN v3 is a little slower than Torch with CuDNN v3, yet the bar for leaf is positioned to the left of the one for Torch, and (2) there's a bar for Leaf with CuDNN v4, but not for Torch.

It's good to see alternatives to Torch, Theano, and TensorFlow, but it's important to be honest with the benchmarks so that people can make informed decisions about which framework to use.

[kibwen]

The graph in the readme is outdated, you can see the version with Torch/CuDNN v4 here: http://autumnai.com/deep-learning-benchmarks

And I don't believe the first point counts as deceptive; the bars are ordered by Forward ms, not by the sum of Forward and Backward. In both CuDNN v3 and v4, Leaf is faster than Torch by that metric (25 vs 28 for v4, 31 vs 33 for v3).

[emcq]

Yes, on their site they post Torch CuDNN v4 as faster than Leaf [0]. Seems exciting for an early release.

Can it get much faster than something like Torch? I would think if CuDNN is doing most of the computation time it would be hard to see big improvements. Perhaps go the route of Neon and tune your GPGPU code like crazy [1, 2], or MXNet and think about distributed computing performance [3].

[0] http://autumnai.com/deep-learning-benchmarks

[1] https://github.com/soumith/convnet-benchmarks

[2] https://github.com/NervanaSystems/neon

[3] http://alex.smola.org/talks/NIPS15.pdf

[jean-]

> Leaf with CuDNN v3 is a little slower than Torch with CuDNN v3, yet the bar for leaf is positioned to the left of the one for Torch

I think that's because they're sorting by forward time rather than forward+backward. That would also explain why in the Alexnet benchmark Tensorflow (cuDNN v4) is to the left of Caffe (cuDNN v3) despite having a much taller bar overall.