[Xeoncross]

https://capillaries.io/ (Cassandra & RabbitMQ) reminds me of https://temporal.io/ (PostgreSQL)

Next up, TemporiarriesLite™

Go + SQLite using https://litestream.io for single-instance, low-power systems or server-less, seldom-used apps that need distributed backups and statefulness.

Jokes aside, these at-least-once operation state managers are really nice and help us avoid adding SQS / NATS / etc.. queues littered all over the place. The focus on data processing by Capillaries is nice. Looking forward to trying it out.

[kleineshertz]

Temporal is a different ecosystem (and a much more ambitious solution), but one of the principles is the same: users want a platform that solves scalability issues and lets them focus on biz logic and customer value.

[lorendsr]

Temporal also shares the principle of being tolerant to database and processing node failures

[lorendsr]

Temporal has different database options: Cassandra, Postgres, MySQL, SQLite.

> source data in CSV files; - Capillaries script (JSON file) that defines the workflow and the transforms; - Python code that performs complex calculations (only if needed).

Temporal is more general purpose: source data anywhere, and you write code to define workflows and transforms instead of JSON, and the code can be in Go/Java/Python/JS/TS/.NET

[kleineshertz]

Right. And this is where our paths go separate ways. As I see it, Capillaries users are not necessarily tech companies and they do not have much appetite for writing and maintaining a lot of code. All they want is to run, say, 50 kinds of workflows on a regular basis and to keep those workflow definitions very formalized and stable. It would be hard to sell the idea of maintaining 50 different codebases to their management.

As for more complex calculations: in Capillaries, Python is not a programming platform, it's just a scripting engine.

[lorendsr]

There are definitely ease of use benefits to more tailored solutions. If workflow definitions are really simple and don't change much, JSON might be easy. Most things I prefer the DX of writing the logic in code. And it wouldn't be 50 different codebases—it would usually be a single codebase with 50 different functions.