[nesarkvechnep]

When I proposed to use RabbitMQ for service integration my colleagues didn't agree to go with it because it "won't scale" and AWS doesn't provide a managed service. Because of an AWS fetish and awful development experience the product won't launch anytime soon.

[pyepye]

AWS released the option to have a managed RabbitMQ broker under Amazon MQ back in November (2020) https://aws.amazon.com/about-aws/whats-new/2020/11/announcin...

[jpalomaki]

One thing to note is the pricing. Clustering is not supported with the micro instance, so if you need it, you must go with at least mq.m5.large [1] and you need three of them, so the total cost is around $620 per month (or closer to $700 for example in eu-central-1).

[1] https://docs.aws.amazon.com/amazon-mq/latest/developer-guide...

[KptMarchewa]

Why not use SQS if you're bound to AWS requirement?

I've found RabbitMQ difficult operationally, and full of footguns (that can make you lose data), so I'm not sure why would you want to use it if you don't already have to.

[dsign]

Not SQS, but Google PubSub: everything was working allright, until one day messages started arriving with horrible latency and vital parts of our service went down. We spent close to two weeks debugging our infrastructure, because _sure it could not be PubSub_. Well, _it was_. I guess nothing too big, a small bit or tweak that changed in their service that didn't propagate to the client library we were using, or god knows what. Now we run RabbitMQ for everything; at least we control our versions and we can confidently blame ourselves when something happens. But RabbitMQ, standalone and clustered, has served us well across different systems for about fifteen years.

[cratermoon]

At a previous employer I had what I thought was close to a slam dunk case for messaging with RabbitMQ. While I'm not the most experienced person in developing distributed messaging systems, and I expected technical objections, the disheartening response, and ultimate reason for rejection, was simply organizational inertia. In short, "you can't use RabbitMQ and messaging for that because we don't use messaging here". I'm sure there was some history that nobody mentioned or some lost institutional knowledge. The "grandma's ham" of systems design.

[zomglings]

I can sympathize with your colleagues. Developing the distributed messaging system is not where the bulk of the cost of running such a system goes - it's the maintenance and debugging that soak up time, money, and tears.

While my past experiences with RabbitMQ in production have been stellar, I can see why a team would be hesitant to add this complexity to their infrastructure.

[cratermoon]

Oh here's the thing though: the company was already using RabbitMQ, as well as Kafka (for other things, mostly data science), and it was all deployed in Kubernetes and replicated across three data centers, including one in Amsterdam. The complexity was already there, but only for a very restricted set of applications.

[datavirtue]

All of these problems sound like they come from developers who don't know how to develop distributed systems. If anything this should be simplifying an otherwise intractible problem. I have seen people overuse messaging and had to fix what I could. I otherwise reach for RMQ as a simple solution to horizontal scaling and I write the software to easily switch brokers (Aws SQS, on prem RMQ, cloud RMQ). It's not something that you choose to adopt and then force developers to make every interaction a distributed message.

[cratermoon]

As I noted, I'm definitely not the most experience in implementing messaging system. Can you recommend some articles or books on distributed systems and messaging that I should know?