[merkleforest]

It would be great if the infra layer can provide some help on automated resource scaling, especially for RAM. The ML solver/tooling layer has also been making progress on this front, for example Dask for limited-RAM pandas, h2o.ai has limited RAM solvers, xgboost has an external memory version, pytorch/tensorflow models are mostly trained on SGD and only needs to load data batch by batch. It's nice that Metaflow can integrate with any python code and thus benefit from all of the efforts made on the solver/tooling layer.