[argonaut]

Just a note: Logistic regression on raw pixels already gets you 88% accuracy. Nearest-neighbors on raw pixels gets you 95% accuracy.

You should not draw too much conclusion from a network with accuracy < 97%, because you probably just have bad hyperparameters (except for conclusions about which hyperparameters you need to tune).

[FigmentEngine]

do you have a source for these? ta

[argonaut]

https://www.google.com/#q=mnist