Hacker News new | ask | show | jobs
by Plugawy 2224 days ago
An application usually has one connection, and many channels. Our pattern is to dedicate one channel for all publishing and then N channels mapped to consumer threads.

You don't have to pool connections as channels are multiplexed by them.

Things to watch out for:

- opening too many channels - these map to Erlang processes and can overwhelm your server if you go over ulimits - sharing consumer channels between threads - you might see weird behavior (e.g. acking wrong messages etc)

We've built own library/framework for creating resilient consumers, and it enforces mapping 1:1 channels and consumer threads, as well as automatic reconnections and channel clean ups.

1 comments

+1 for everything that's been said. Another thing to consider is message throughput, if that's a concern. In the case of multiple channels per single connection, note that a connection is a single TCP connection such that multiple channels contend for the TCP stream. At the same time, connections aren't completely free either.

The general takeaway from this should be: if you've got a particular stream of messages (either a producer or a consumer) that pushes many thousands or even tens of thousands of messages per second, use a separate TCP connection. For anything else that is slower (dozens of messages per second), multiple channels on the same connection work great.

One last consideration is that when a given channel misbehaves or you perform an operation that the broker doesn't like, the only recovery that I've seen is to shut down the entire connection which can affect others channels on the same connection.