Just a note: Logistic regression on raw pixels already gets you 88% accuracy. Nearest-neighbors on raw pixels gets you 95% accuracy.
You should not draw too much conclusion from a network with accuracy < 97%, because you probably just have bad hyperparameters (except for conclusions about which hyperparameters you need to tune).
There is a considerable lag when I drag and release the slider. I assume you are doing some intense computation, but perhaps allow the UI to be responsive while you are doing so?
This was the first thing I noticed too. What's the point of a slider if I can't watch it change as I slide? It'd make more sense to just show me a series of images or possibly allow me to select what I'm looking at based on some kind of button or something. The slider interface makes it feel like something that you should see changing live.
That's a good point. I tried some light optimizations using ShouldComponentUpdate in React, but I believe the bottleneck is a synchronous blocking call in the D3 rendering process. Larger networks write up 25,000 SVG elements, and I was not sure how to significantly improve rendering speed. Someone suggested Canvas with D3 could speed rendering up?
If possible, send it off to a web worker? That free's up the UI thread
Edit: So it's the rendering that's hard on the CPU. Canvas would probably improve performance. Also, the graphic is so simple and there doesn't seem to be any event listeners on the edges themselves, that converting should be trivial :)
For the most part you're not doing live animations, statically drawing on a 3 canvases would give you what you want (without huge overhead of DOM nodes and simplicity of 2D canvas painting). Web workers aren't going to do much since most of your lag is coming from way too much DOM (causing layout, repaints to take forever).
In my experience spinners work great for async calls, but not so great for blocking synchronous computations. There is probably something that could be done though with a little bit of creativity though...
> Most networks gained a large percentage of their final accuracy in just one epoch. And it usually was the case that a higher accuracy in the first epoch meant a higher final accuracy.
This sounds to me like learning was just crawling to local optimum not actually exploring or making any breakthrough in understanding of the domain.
The largest float in the output layer (while the graph is yellow) is the activation. The largest activation in the final layer is the network's "guess". The guess is the index of the last layer, which corresponds to a particular digit.
Awesome, I just flipped that switch at the top and can see how it calculates the individual handwritten inputs. Great demo!
So I guess during training you're telling it that correct answers should be 1 and the incorrect answers should be 0.
Do the encoding choices that you make regarding the input / output of a neural network influence its performance at all? Maybe for MNIST the way you have it is the most common approach?
Usually the number of nodes in the input and output layers don't affect things all that much. They are relatively set based on the problem. The number of nodes in the hidden layers, the number of hidden layers, and various other parameters such as cost and activation functions, are mostly what you use to tune performance.
Has anyone tried training a neural network on anything that isn't the NIST digits? I've seen that one so done to death and a dearth of other examples that I'm starting to get skeptical that it would work on any other cases.
Or is it that the data is expensive and the data "scientist" is cheap?
You should not draw too much conclusion from a network with accuracy < 97%, because you probably just have bad hyperparameters (except for conclusions about which hyperparameters you need to tune).