[mind-blight]

This seems really cool. I've just been running into scenarios where this kind of durable execution is really helpful. I've been doing basic things with RabbitMQ plus a job server, but there are definitely limitations.

I'd love to hear from folks who have experience using things like Beam or Spark. I've if the biggest pain points I've encountered is that there are definitely dozens of "mature" products to solve this problem that all different slightly in their setup and tradeoffs

[robertlagrant]

I always thought Temporal[0] would be a brilliant choice for that sort of durable processing.

[0] https://temporal.io

[dangoodmanUT]

Temporal is great until it's not XD

[nogridbag]

Care to elaborate or link to your thoughts? Temporal has always been on my radar. I wanted to do a deeper dive, but the cost seems pretty high for the managed service.

[jahewson]

Good concept, buggy implementation. That’s not ok for a distributed system that glues together important pieces of your product.

[robertlagrant]

I'd also love to know more. What sort of thing is an issue?

[chipdart]

> This seems really cool. I've just been running into scenarios where this kind of durable execution is really helpful.

You might find Azure's Durable Functions right up your alley. With durable functions you can break away workflows into activities which are invoked from orchestrator functions or other activities like regular functions, but the runtime handles the orchestration and state machine updates.