What's the advantage for your situation of publishing to SNS which is in turn publishing to SQS vs. just publishing directly to SQS? Are you forking to multiple SQS queues?
Have you looked into a "real" messaging queue for this, like RabbitMQ or another AMQP based system for this? Or are you bound to SNS/SQS for a particular reason?
"Real messaging" aside, this is an interesting question. It'd be great to hear from iain_hecker if they considered using RabbitMQ (or similar) and if yes, why SQS/SNS were chosen.
Also, I'd be interested to read more about how you handle authentication across web/mobile. Thank you for the blog post and for taking the time to answer questions here. :-)
We have considered stuff like RabbitMQ. They would be excellent solutions too and we are working on a part of our architecture with MQTT/protobuf (more on that later, I'm sure). We went with SNS/SQS because we don't have to build a cluster with that. Building clusters is hard and Amazon takes care of that. Since we were already using a bunch of AWS products, this was a nice fit and we're quite happy with it.
Authentication is a good idea for an article too. Thanks :)
Sorry, my question was a bit vague. Generally SQS works with polling, and once a message has been pulled from a queue it is not available to other viewers any longer. This makes it hard to create a system where you have an unknown number of clients that may or may not be interested in certain queues, or even in messages of certain types.
RabbitMQ (and other AMQP based queueing systems) have built-in solutions for this that SQS doesn't offer. Karma solved it by sending the same message to a number of different queues, but this makes scaling to more clients harder, and reduces flexibility in subscribing clients to certain message types.
That's where SNS comes in. We publish to SNS which will fan out to multiple SQS queues. When a new service is built, it says to SNS "subscribe my queue to these topics". The publisher of the event doesn't know about the subscribers.