[bjt]

I thought the same thing about Rabbit. PinLater's ACK system feels like a questionable reinvention of AMQP. I'm probably missing something.

Rabbit's advertised throughput doesn't reach 100,000/sec, but you should be able to shard your way around that. Or use Apache Kafka, which has the sharding built in.

[fintler]

Just for comparison, LinkedIn is doing about 2.5 million messages per second (on average per day) with Apache Kafka.

http://www.slideshare.net/ToddPalino/enterprise-kafka-kafka-...

[hatred]

Precisely, this is the same query I had when I glanced down the post. It will be great if the author or someone from the team respond to the various trade-offs and why they did not go ahead with Kafka ?

[omrvp]

(I am the author of the blog post). I posted a reply below earlier this morning that hopefully addresses your question.

[bigtones]

Looks like they needed to have built-in support for time based scheduled execution of jobs at a specific time in the future. Rabbit MQ does not have that (that I'm aware of) without doing some horrible hacks around one queue per message.

[kondro]

ActiveMQ's had that for a couple of versions though. Another highly performant/available, atomic, consistent, durable and mature queueing server.

[cyounkins]

Correct. Celery's implementation dequeues but doesn't acknowledge the task, holding it until the timer is reached.