[eli_gottlieb]

I understand stochastic gradient descent just fine, but you need to reverse your thought here. Just because something is trivial for us doesn't mean it lacks inherent difficulty. After all, if it was inherently trivial, pretty much everyone would be able to actually derive and implement the algorithm, which the vast majority of the human race can't.