[datalopers]

Chiming in to concur. Redis is amazing and simple software. You can use a managed service like Elasticache or install the binary on a VM instance. Folks using a relational database for a job queue or a cache when an infinitely simpler and more appropriate tool is available are just making poor technical decisions.

[adamckay]

But it's not simpler when you consider all the things you've got to do around and after installing that binary on a VM instance. Consider the overhead of managing it - monitoring it, updating it.

Failover when the node dies. Clustering for high availability?

Backups? For a cache, probably not, for a job queue broker, probably necessary.

Making sure your app deals with inserting into Redis on successful transactions and not when a transaction is rolled back.

Getting up and running can be fairly painless, staying running on all edge cases and handling partial failures is what gets you.

[datalopers]

I’m confused the scenario you have where a) a singular postgres install which does everything is acceptable vs b) as soon as redis comes into the picture, suddenly you need HA and monitoring and apparently running transactions with full ACID integrity?

It’s just a nonsensical and unfair comparison. You can run a single Redis instance with normal rdb disk syncs and don’t ever update it for years on end without issue. Is that guaranteed resilient? Absolutely not, but that’s not the scenario in discussion. We’re talking about the context of a bootstrap/MVP scenario, not an enterprise setup.

I’d take a single-node redis job queue everytime over a HA citus/postgres cluster improperly acting as a queue.

[tmd83]

I think the point is they have a Postgres server running anyway as the datastore and the job queue being in Postgres gives you HA, backup and Transaction for free. I think Redis in particular won't give you transaction right?

Needing Transactional semantics for jobs alongside an application operation makes a lot of simpler queue/tool choices difficult.

[julik]

Thank you for the nuanced assessment, I would tend to disagree still.