[scosman]

Love this space. I’ve build a few versions of this in go, glad to see some go native frameworks.

Would also love to see some examples.

Concerns I’ve had to deal with in the past (if these are solved, examples would go a long way):

- max queue size between elements is important. Don’t want the database reader role to outrun the workers, but also never want them desaturated

- some dynamic limits for local to a role (limit DB readers to limit connection pool) and global (how much cpu and memory share does a job need before starting)

[harshadmanglani]

Thanks for the inputs! I'll add some examples in the documentation.

- with control on the database reader yourself, i think you should be able to find a way around saturation/desaturation?

- again, the fairly certain you can limit DB readers in the DataStore interface you pass to the orchestrator. I'll think about the cpu and memory share and if there's a way to expose that.

A major concern I've always had with workflow orchestrators is the versioning of workflows. Think about long running workflows (>2 days). If you change your workflow logic, what happens to the existing ones that haven't completed yet?

Everyone handles this differently, and I've been thinking about a generic way to do this. Thoughts?