[mlthoughts2018]

PyTorch is catching up to TensorFlow, but still has a far way to go especially compared with the user experience of Keras with TensorFlow.

In PyTorch you have to do quite a bit more work specifying gradients, zeroing the gradients between update steps, specifically describing the operations of the backward pass in a custom layer, even in cases when it could just be inferred by the computation graph library as in TensorFlow.

I’ve heard anecdata that PyTorch performance can be better, but I think it’s still a case-by-case topic, and both libraries are still changing rapidly.

I attribute the switch from Lua Torch to PyTorch as just an obvious effect based on how bad of a language Lua is for the type of interactive and experimental programming needed for this type of work. Lua is a good language for some applications, but not this — and the value of spending effort to build the primary Torch frontend in Python became obvious quickly.