I was confused with the difference between an SVM and a neural network. Easy mistake I guess. The whole goal of this project was for educational purposes, so im still happy with the outcome.
OMG, how embarrassing for OP. This is what scares me about technical blogging. Messing up unknowingly in an area I'm not experienced in and getting scathing critiques from my fellow hackers. Keep your chin up OP and next time remember to do your homework. +1 for the effort anyways.
if you want to learn neural nets check out Karpathy's class (cs231n.github.io) and do the assignments. making a github repo and HN post about using neural networks is false self-advertising and illegitimatizes those of us who know what we are talking about.