[glogla]

This is why I though the shift from "orchestrate jobs" to "keep track of state of assets" that Dagster is trying to do is pretty important. But it sems it might not be enough - it still keeps clunky (pythonic) interface and I don't know how well it does multi-tenancy.

[sethjr5rtfgh]

Could you clarify what "keep track of state of assets" means?

[glogla]

https://www.dagster.io/blog/software-defined-assets

Of course, nothing stops Airflow or other tools fron thinking this way as well.

[geoffjentry]

It's a mindset shift to a more declarative model. The idea has also popped up in other niche orchestrators.

This is an oversimplification but IMO the easiest way of picturing it is instead thinking of defining your graph as a forward moving thing w/ the orchestrator telling things they can be run you shift to defining your graph nodes to know their dependencies and they let the orchestrator know when they're runnable.