[bluemania]

Thanks for submitting, this is going to come in handy for my education! I have an assignment next week which requires creating a neural network, then substituting various optimizers in place of backpropagation and comparing performance over iterations. Have found that simple numpy based NN's are easier to examine and connect the changes with the theory, this guide looks to help further with this understanding!

[dnautics]

That sounds really fun! Do you get to pick optimizers? Obviously direct gradient is a thing (calculate the partial for every parameter) but also particle swarm optimization might be cool to profile.