[brightball]

Is there a big advantage over Redis? Just coming from the Ruby world I know Sidekiq is extremely popular as a Redis backed queue. I know there's a RabbitMQ backed queue called Sneakers that's gained a lot of popularity as well though.

Just wondering what the biggest selling points are for somebody making a decision?

[abe94]

We switched from a redis backed queue (bullmq) to rabbit at our company (https://github.com/laudspeaker/laudspeaker) when we hit speed issues . We found bull had a much lower throughput, after a lot of tuning compared even to rabbit out of the box.

We needed features like job priorities, and complex routing logic and on top of that our software needs to send millions of messages a minute. What we miss is a nice UI to see the state of jobs, and monitor for problems more easily like: https://taskforce.sh/

[tejinderss]

At my current place, using Redis with celery is becoming bottleneck for number of concurrent connections it can hold. We are using 1000 workers and start seeing issues (ceiling is 10k connections in Redis); apparently Celery creates huge number of connections. We are considering moving to RabbitMQ for same reason.

[zo1]

Celery is an over engineered pile.. Rather move lower down the stack, with your first step being Kombu which powers celery under the hood. It's oddly "configurable" so if you need to optimize and adjust connections this is where you should go, and it's pretty interchangeable between Redis and AMQP. Really, almost a drop in replacement that should just be a config change.

[cloverich]

Been a while since I last used, but IIRC Rabbit is much more featureful than Redis, and has built-in much of what you'd get from Redis + Sidekiq. Their concepts[1] docs provides a good overview. This may be too broad a stroke but what I liked best about it was we used it for our centralized pub sub system, and although we had services in multiple languages / frameworks, they connected to each other through Rabbit, which saved us from having to setup a bunch of different / incompatible job processing systems. Like Redis its battle tested so you know what you are getting complexity / scale wise. At my current gig we use the typical Rails / Sidekiq setup and though it works fine I definitely find myself missing Rabbit. (But in my head Redis / Rabbit have only some overlap, and seeing both at the same company would seem totally normal)

[1]: https://www.rabbitmq.com/tutorials/amqp-concepts

[bankcust08385]

Redis is slow and doesn't scale out well. Couchbase used to be a decent choice but it went commercial pseudo-FOSS.