Hacker News new | ask | show | jobs
by SmirkingRevenge 2957 days ago
I upvoted your question - not sure why its been downvoted, as it seems like a good-faith question, and not to mention - a very reasonable question.

Subtle differences in the semantics of pub/sub and message passing services can have really significant consequences for their use-cases.

Google Pubsub is at-least-once delivery, and best-effort ordering. That means any consumer pipelines need to be able tolerate duplicates and out-of-order messages - in many cases, that's not trivial to handle. If the order of your messages isn't that important, and if you can make the operations of consumers idempotent - pub/sub is awesome. But more often than not, it becomes just another message passing service to add to your plethora of message passing services, because its limitations can't extend to all of your use-cases. (I really want one ring to rule them all - Kafka gets the closest, IMHO)

Kafka is an basically pub/sub on top of an ordered, append-only log, and consumers read the log stream very much like a single process reads/seeks on a file handle - using offsets. Given infinite storage - your entire data stream can be replayed, and any data can be recreated from scratch - thats a pretty awesome thing.

1 comments

>> I really want one ring to rule them all

Try Apache Pulsar: https://pulsar.incubator.apache.org/