[mlthoughts2018]

I think you flipped my comment around. I’m saying that the number one defining requirement of model reproducibility tooling is that it does force version control / code review.

It should force the concept of “running an experiment” to be just another instance of a deployment. Any part of running an experiment that happens outside of the scope of that, such as with “mlflow run ...” for example, is immediately violating the most basic property of the whole thing (I guess unless “mlflow run ...” is hacked to perform actual production deployments of all types of programs).