[mattrasmus]

Completely agree with this view. I wish I had seen this thesis earlier when developing redun (https://github.com/insitro/redun/). Looks like a lot of very useful ideas for defining such a system.

We based redun's execution model on very similar ideas of functional programming and graph reduction. In addition, we made it work as an embedded DSL within Python, so one can easily use all the typical data science and ML libraries in a workflow. This has been very helpful for building biotech workflows (genomics, imaging, chem).

I am a bit surprised why many workflow systems shy away from full turing-complete. You usually don't need to trade that away for automatic parallelism, caching, etc.