Hacker News new | ask | show | jobs
by jdunck 4803 days ago
My recc after a lot of heavy use is: RabbitMQ for the queue, Redis for the result store.

RabbitMQ for the result store is madness.

1 comments

why not just use redis for both - just curious ?
Redis does OK as a queue, but not as well as rabbit.

For a start, rabbit handles OOM better. It has lower per-message overhead. It has a more flexible queueing model in general so that you can use it for both work queue-y stuff and other queuing needs. It's interesting how often queues seem handy once they are easy to use.

Then there's the fact that redis is good for lots of things other than queues, and you'll be tempted to use it for that, and that will crowd the queue use. If you hold a transaction on redis or do a large set intersection or run a lua script, you block everything else, including your fanned-out celery worker pool.