[awinder]

AWS has sqs & kinesis which are much better queueing options in that scenario where you’re in AWS doing new dev and can pick a technology. This is more likely to do with opening doors for large and complex applications that can’t be rewritten to come into the AWS cloud. Unsexy stuff but there’s some fun engineering to be done in that realm, if you find like puzzles and shitshows to be fun anyways :-)

[twelve40]

how are those much better? if you use SQS and you write your code for it, then you are stuck on a proprietary platform. Also, SQS is super-basic and actually requires a bunch of code to do anything beyond trivial - although yes, it seems reliable and well-supported, at least from my experience. I was actually really waiting for AWS to support Rabbit since it seems to hit the right combo of features, usability and platform independence for me, and it looked friendlier than ActiveMQ.

[nijave]

If you're using a decent framework there's a good chance it already does most of the work for you. With Ruby on Rails, there's the ActiveJob abstraction which you can hook up to different backends like SQS or Redis with a few lines of code. In addition, AWS has a lot of out of the box integration with SNS and SQS for other services like Cloudwatch and Lambda (in fact, lambda you don't need any special code).

If you have a Lambda function processing SQS messages they just get dumped in your handler method and it your function runs successfully they get automatically removed from the q. If your lambda fails, the message reappears after the visibility timeout out subject to your redrive policy

[twelve40]

I'm referring to things like topics, multiple consumers, routing, etc etc that are not even possible with SQS and once you grow into a need for those, SQS stops being adequate no matter what library you use for it.

[constantine42]

"sqs & kinesis"? These are two vastly different queuing systems. It's like saying "SSDs and Tape" are a better storage system than X.

[manigandham]

Neither SQS nor Kinesis support the same functionality as RabbitMQ.

[avinassh]

> AWS has sqs & kinesis which are much better queueing options in that scenario where you’re in AWS doing new dev and can pick a technology.

Why are they better?

[whatsmyusername]

They aren't. They're technically more correct but not always the practical best choice.

RabbitMQ is a smart play as Rabbit is very easy to use, understand, and troubleshoot at the low end (which is where I suspect the vast majority of queue systems live).

It also has a feature which is actually really hard to do (and sqs doesn't do). Guaranteed delivery of a message once.

That was THE reason we never migrated to SQS, there are scenarios where SQS can double deliver. Our codebase was built up from nothing over time and couldn't gracefully handle double delivery of messages in all scenarios. We could have refactored, but it wasn't worth the work when we were already doing a half billion in revenue without getting even close to the limitations of rabbit AND were close to selling (which we ultimately did).

AWS is great at selling multiple slight variations of the same product. If you look you can usually find ONE variation that works for you. The real test will be if the billing isn't garbage (garbage billing is why we didn't use their other AMQP service and part of the reason why we don't use things like EKS or Managed SFTP despite having the need).

[eloff]

> Guaranteed delivery of a message once

That flies in the face of my distributed systems knowledge. It's not possible in some failure cases.

If your acknowledgement of a message gets lost (because either server involved or the pipes in-between fail) you've processed the message already but the queue server will think you haven't. It either has to resend it (duplicate delivery) or it ignores acknowledgements all together (drops messages that it sent you, but you didn't process - maybe because your server failed.) So the choice when there is a failure in the system is between at least once or at most once - exactly once cannot be guaranteed.

I'm not aware of any way around that predicament.

[whatsmyusername]

You are correct, a better description is that their path to 'deliver exactly once to the best of your ability' is clearer.

If I remember correctly SQS is hard limited to a fairly short timeout to requeue messages delivered but not acked. In rabbit it's much more configurable.

Also regular rabbit hosts support the kludge pattern of, 'just run one host and accept if it goes poof you can lose messages,' which is useful if you don't want to bother with the complexity of clustering or are on a shoe string budget.

Lastly you get a nice user interface with the management plugin and you can stand it up locally with docker compose (without depending on AWS for dev or any of the 'aws but on your laptop' solutions).

[eloff]

Yeah, those are nice features to have. Plus you don't get the platform lock in.

[whatsmyusername]

Normally I don't give a crap about vendor lock-in, but this would be an exception where I'd agree with you.

Though most people are just going to use a framework plugin to manage the messaging layer, so what's behind that is largely irrelevant.

[brightonship]

SQS also supports FIFO queues, which have once-only delivery and ordering. Any reason those didn't work for you?

[whatsmyusername]

Aren't they expensive with performance limitations?

Yes we could do that, but we had already been using rabbit in a bunch of places. It made no sense to change it.

[awinder]

They’ve been around for quite some time so they have a much wider customer base & bigger teams supporting them. AWS services can be a bit choppy in the beginning so imo, especially with queues, I’d wait for it to bake.

[pmiller2]

I agree. SQS has done nothing but improve over time.