Hacker News new | ask | show | jobs
by martinpeck 503 days ago
I think the example given in this blog post might need a "health warning" that steps should, generally, be doing more than just printing "hello".

I can imagine that the reads and writes to Postgres for a large number of workflows, each with a large number of small steps called in a tight loop, would cause some significant performance problems.

The examples given on their main site are a little more meaningful.

2 comments

Yes, that's totally fair. Usually, a step is a meaningful unit of work, such as a API call that performs an external state modification. Because each step is a fair chunk of work, and the overhead is just one write per step, this scales well in practice--as well as Postgres scales, up to 10K+ operations/second.
I feel like you can generalise this to any transactional key value system, which can scale better.
Your intuition is good - having worked with something similar to this, it works great but does not scale very well. The step journaling is pretty brutal to postgres/rdbms , and you hit vertical scaling limits quicker than you would like