Great idea but however equally great caveat - it's just for (forward) inference. Unix pipelines are fundamentally one way and this approach won't work for back propagation.
I don’t see any reason you couldn’t just spit out the output and the derivative of the layer output with respect to the weights, then multiply and carry these all the way down. Then if you have a loss function at the end you have the gradient. Probably this project is for fun and not scale so it’s fine. But then you need to think about changing the weights on every layer based on the optimization