Hacker News new | ask | show | jobs
by rockwotj 486 days ago
The kafka protocol is a distributed write ahead log. If you want a job queue you need to build something on top of that, it’s a pretty low level primative.
2 comments

Not for long. An early access version of KIP-932 Queues for Kafka will be released in 4.0 in a few weeks.

https://cwiki.apache.org/confluence/display/KAFKA/KIP-932%3A...

Why does everybody keep missing this point? I don’t know.
There's a wonderful Kafka Children's book that I always suggest every team I work with read: https://www.gentlydownthe.stream/

The way I describe Kafka is, "an event has transpired... sometimes you care, and choose to take an action based on that event"

The way I describe RabbitMQ is, "there's a new ticket in the lineup... it needs to be grabbed for action or left in the lineup... or discarded"

Definitely not perfect analogies. But they get the point across that Kafka is designed to be reactive and message queues/job queues are meant to be more imperative.

Your two-sentence description is excellent. That book, not so much.
I suppose that's fair.