[maxbond]

I don't think "much of the magic is X" implies "only X matters." A lot of the magic of a NN framework is in getting good enough performance that the problem becomes tractable. PyTorch and Tensorflow give you a lot of out of the box abstractions, which is great, but the most important thing they do is compile your model to run on the GPU so you can do an epoch in 20m instead of 2hrs.