Hacker News new | ask | show | jobs
by jeffasinger 2593 days ago
Often times these "Service Bus" type frameworks implement request/reply by creating a queue per host, and in every message adding envelope information that includes the queue that the reply should be sent to. This does allow a request/reply pattern. I've never actually done this, but I imagine in environments where reliability is more important than latency, it probably works pretty well.

One minor nitpick. The patterns you're talking about aren't typically implemented on Kinesis, since there's no concept of topics there. On AWS, you could do something like that with SNS + SQS.

1 comments

I know the least about Kinesis, but my assumption is that it still has an OK-like response to any batch of events you send to it, does it not?

I think you might be misunderstanding me. Obviously I know about topics. In Kafka and EventHubs (where really you have a namespace containing multiple event hubs corresponding to topics) at least, there is no reply pattern whatsoever beyond an OK. I still don't know which specific distributed message queue platforms you and others are referring to that implement this pattern.

Sorry if I wasn't clear. You aren't relying on just the ack semantics, you're building bidirectional communication on top of software, in a way that's potentially a little unnatural.

Here's an example of how you'd accomplish this with Kafka. Whenever a new instance launches, it could generate a UUID, and create a Kafka topic with a matching name. Any messages it sends that it expects a response to could include a "reply-to" field, with this UUID. When something processes the request, they publish a message to that Kafka topic.

Essentially when people are talking about "Service Bussses", they're talking about frameworks that implement this and other similar patterns on top of generic queues like rabbitmq, msmq and sqs. One such framework I've personally used is MassTransit.

It's also trivially easy to extend this to Kinesis with a lambda.
Sort of how any Turing machine can solve any problem solvable by any other Turing machine, almost any communication system can be used to implement other patterns of communication. You combine the primitives of the base to provide the structure you want above it.