Hacker News new | ask | show | jobs
by Roodgorf 3186 days ago
In most cases I think SNS is closer to what I want: multiple producers feeding into a single pipeline that notifies its consumer as soon as possible. The main thing from SQS is the ability to retry a failed message.

If SQS were event based, you could potentially try to process a message as soon as it was received and then have a batch job to reprocess any messages that have been left in the queue however often you like. The closest to this we got was by using SNS and feeding it into SQS, or even a table somewhere, but that gets sort of messy pretty quickly and didn't seem worth the extra infrastructure.

Edit: I didn't really respond to the suggestion of Kinesis here. I haven't done much with Kinesis, but I get the sense it's overkill for the use cases I'm thinking about. I don't plan to have a large amount of producers and don't expect them to have a consistent or large stream of data. If I'm wrong in that being the main use case for Kinesis please correct me.

1 comments

I haven't used kinesis myself, I've just heard that it's "SQS with the FIFO issue fixed", and wondered if it might also be suitable for you. Judging from the other comments, it wouldn't be a silver bullet, though.
Speak of the devil - I've just had to create a new SQS queue just now (first time in months), and there's an option to make it FIFO.