[theschreon]

You could try the following improvements to speed up neural network training:

- Resilient Propagation (RPROP), it significantly speeds up training for full batch learning: http://davinci.fmph.uniba.sk/~uhliarik4/recognition/resource...

- RMSProp, introduced by Geoffrey Hinton, also speeds up training but can also be used for mini-batch learning: https://class.coursera.org/neuralnets-2012-001/lecture/67 (sign up to view the video)

Please consider more datasets when benchmarking methods:

- MNIST ( 70k 28x28 pixel images of handwritten digits ): http://yann.lecun.com/exdb/mnist/ . There are several wrappers for Python on github.

- UCI Machine Learning Repository: http://archive.ics.uci.edu/ml/datasets.html

[dfrodriguez143]

Definitely a lot to read and improvements to make. I will probably do a more complete benchmark with more datasets on a later post.

Thanks for the suggestions.

[benhamner]

You may be interested in this ICML 2006 paper, which empirically compared many standard algorithms across a combination of metrics and UCI datasets - http://www.cs.cornell.edu/~caruana/ctp/ct.papers/caruana.icm...