Wow. This paper is _very_ well written. I am not an expert in neural networks, but the concepts were very well described and diagramed and I was able to follow along.
I would read a whole book about neural networks written in this style.
Normally, ML training is via back propagation, which is a synchronous technique. If you try to trivially parallelize, it doesn't work, for reasons (tm).
This lets you train a machine learning model of arbitrary size (bigger than can fit on a GPU, or even a multigpu node) using an actor-based distributed technique. There is a slight training cycles count penalty but it's way less than the cost of coordination.
I would read a whole book about neural networks written in this style.