Hacker News new | ask | show | jobs
by joowani 3515 days ago
RQ is certainly useful (with finer control over messages), but we've been having a lot of problems with it lately in production due to it being memory bound (Redis). Improper code deploys or insufficient/stuck workers would quickly explode the queues and make the broker go oom in matter of hours. Memory is also very expensive. With KQ/Kafka I was hoping it would provide us with a lot more to buffer for human errors and scale better.
1 comments

Ok, that makes sense. I've never dealt with the scale you are talking about so have yet to run into these issues with redis. It may be best to put this info in the readme so people can decide for themselves when to use RQ or KQ.