[tweaktastic]

I agree with author on the point that queues are best to handle spikes. But when you are talking about scale dealing with millions of messages in short span of time for extended period, queues can never remain empty. Empty queues in this situation would mean that your concurrency is high. When your concurrency is high, this would result in too many concurrent connections to your database which wouldend upcreating bottlenecks for your system.

The system should be designed to handle as many messages as possible ensuring that your resources don't get over burdened. Empty queues where your messages keep getting rejected because your resources are being hammered is not necessarily an efficient system.

[roflyear]

> When your concurrency is high, this would result in too many concurrent connections to your database which wouldend upcreating bottlenecks for your system.

Can you expand on this?

[dmoy]

If you have a polling queue, as the number of workers grows too large, the overhead of maintaining connections from (worker) -> (stuff in the middle) -> (db or whatever keeps queue state) becomes too high. In the extreme case, you can get into a point of diminishing returns, where adding more workers hurts both throughput and latency in a bad way.

That is - initially you can decrease latency by sacrificing utilization and just adding more workers that usually sit idle. This increases throughput during heavy load, and decreases latency. Until you can't, because the overhead from all the extra workers causes a slowdown up the stack (either the controller that's handling messages, or the db itself that's got the queue state, or whatever is up there). Then when you throw more overhead, you increase latency and decrease throughput, because of the extra time wasted on overhead.

It depends on a lot of different variables.

If you have work items that vary significantly in cost, it can add even more problems (i.e. if your queue items can vary 3+ orders of magnitude in processing cost).

[roflyear]

Well yes but you can solve that in a lot of ways.