[jamesblonde]

This is a very good way to get started building ML pipelines. When you do it at scale, you often need to use a broader range of tools. Here's how we do it in Hopsworks with Python the whole way (using Airflow to orchestrate the different steps):

https://hops.readthedocs.io/en/latest/hopsml/hopsML.html

[Octokat]

Your pipeline design looks pretty sweet!