[rspeer]

I use Snakemake [1], a parallel make system for data, designed around pattern-matching rules. The rules are either shell commands or Python 3 code.

I settled on it after originally using make, getting frustrated with the crazy work-arounds I needed to implement because it doesn't understand build steps with multiple outputs, switching to Ninja where you have to construct the dependency tree yourself, and finally ending up on Snakemake which does everything I need.

[1] https://snakemake.readthedocs.io/en/stable/

[reacharavindh]

Thank you for sharing this information about snakemake. I administer a cluster for a group of geneticists. I'll try to get them to use it for their publications to make their results easily reproducible by others.