[adamcharnock]

Something about queuing systems that often gets me is that they can start to seem like the wrong abstraction as soon as one has tasks that enqueue additional tasks. Particularly when features start growing, and double particularly when modelling business processes.

This is because the code enqueuing the task needs to be aware of what happens next, which breaks separation of concerns. Why should the user sign-up code have to know that a report generation job now needs queuing?

Really what starts to make more sense to me is to fire off events. Code can say, "this thing just happened", and let other code decide if it wants to listen. When then makes it an event stream rather than a queue, with consumer groups at al.

I made the (now unmaintained) project https://lightbus.org around this, and it did work really well for our use case. Hopefully someone has now created something better.

So I'd say this: before grabbing for a task queue, take a moment to think about what you're actually modelling. But be careful of the event streaming rabbit-hole!

[globular-toast]

They're not mutually exclusive. Nothing about "event driven" means async. I have an event driven modular monolith and all events are handled synchronously. It's up to the receiver to queue a task if it needs to, so context boundaries are not crossed.

[halfcat]

It sounds like the need you’re describing is not event-driven, but more like orchestration where the orchestrator is aware of dependencies between tasks and runs them in the right order (it builds a DAG, a directed acyclic graph). Tools like Airflow do this.