[pdx]

    The shipment app listens to the messaging system, sees an 
    order take place, looks at the details, and says, "Okay, 
    I need to send two boxes to this person." Any other 
    services interested in an order happening can do 
    whatever they need to with the event in their own 
    queues, and the store API doesn't need to worry about 
    it.

I'm curious how you guarantee that all the systems that need to see an event, will actually see it, before it is removed from the event queue. I assume the shipment app, in your example, is responsible for removing the event from the queue. So, what if it removes it before the mailer app or the "make cash register sound" app, sees the event?

[swanson]

I believe each service has it's own queue, so producing an event would "fan-out" and add the event to each queue. That was my interpretation of why they were using both SNS and SQS - but I could be mistaken. If a particular service is not interested in a given event, it can just pluck it out of the queue and move onto processing the next message.

[iain_hecker]

Each app gets their own queue. So there is a queue for the shipment app and for the mailer app. When an event is published to SNS, it gets added to both queues.

[pdx]

I see. So you never add a listener, without also modifying the emitter to send to an additional queue.

[futurepaul]

We don't change the emitter, the listener changes the SNS config. The emitter publishes to SQS under a certain topic. Listeners that interested in that topic subscribe to it by changing the SNS config, basically saying "give me a copy of those events too and put it onto my own queue". SNS copies the messages to every queue.

[lobster_johnson]

SQS is very simple and the solution here requires that the publisher knows all the listeners.

If you use RabbitMQ (which are we using in our own microservice architecture), you can use fanout exchanges based on topics to accomplish the same thing more elegantly.

In this topology, every message sent to that exchange gets copied to any queue bound to that exchange.

This system also supports routing via "topics", which are paths that support wildcards; the publisher can publish to "foo.bar.baz", and queues can bind to the exchange using the routing key "foo.*.baz", for example.

We use this to listen to specific events, eg. a specific app is associated with content under the path "someapp.someclient". A data store publishes modification events with "create", "update" or "delete" followed by the path; so the the app, to get the stream of updates, simply listens to "create.someapp.someclient.#", "update.someapp.someclient.#" and "delete.someapp.someclient.#".

[a-priori]

You can use SNS to act as a fanout exchange. Then you just need to subscribe all your SQS queues to the SNS topic. Events get pushed to the SNS topic, and SNS will distribute them to all the queues for you with the pusher needing to know anything about the subscribers.