Hacker News new | ask | show | jobs
by tiagod 2164 days ago
I've been using temporal.io, which afaik is developed by some of the people who built SWF (and Cadence at Uber), and I think it's great. I'll probably use it for a ton of stuff in the future.
3 comments

We (at Banzai Cloud) use Cadence for managing infrastructure. Workflow as code is a huge thing for us, because we can use the official SDKs for cloud providers, but still have the flexibility of implementing our own logic easily and in a testable way.

Temporal is even better as it replaces Thrift with gRPC and adds tons of improvements.

We have a bunch of services which orchestrate long running workflows so temporal is quite interesting to me. I've been lurking in the temporal slack for a while (and before that in the cadence slack).

Would be interested to know what kinds of things you're using it for/what scale/which language client?

I work for an accounting company with a few hundred workers, who every month had to submit a lot of files, and collect a lot of documents from human-only websites.

About two years ago I built a custom workflow engine in python where I had a few primitives (filesystem, database access, browser (puppeteer scripts), etc), and would define a workflow using those primitives using a YAML DSL.

In a couple weeks I ported the whole thing to Temporal (Java client), where each of those primitive groups is a different activity, which runs in a separate worker. I have a browser worker, which runs in the only servers that have internet access, a filesystem worker, which is on a Windows machine serving as an entrypoint to the company's windows filesystem, a database worker, etc.

This is all very scalable, and I can run as many of each worker and the engine itself as I like.

Currently the scale is not too big though. I launch a few "main" workflows in certain days of the month that will then spawn up to 4 thousand child workflows which run simultaneously with no problem.

This looks awesome, but I don't understand what this has that airflow, prefect or luigi don't have?
Scale, interactivity, and durability are certainly part of this discussion. You can use Cadence/Temporal for purposes you just really couldn't use Airflow for. For instance, you could implement this entire actor-based auction example from Akka with Cadence/Temporal https://doc.akka.io/docs/akka-enhancements/current/persisten... There is absolutely no way you could do that with Airflow.

To understand why Cadence is so useful, you have to go a bit beyond the traditional ETL scheduling usages IMO.

In SWF/Durable Functions/Cadence/Temporal, The entire workflow is defined as (deterministic) code, which is orchestrated by the engine.
As as opposed to... having undeterministic code? Workflow as code is also the theme of airflow, luigi and prefect.
With Temporal/Cadence you write (mostly) plain code to implement your business logic rather than using code to define DAGs.

This discussion has more on Airflow vs Cadence: https://news.ycombinator.com/item?id=19732447