[anthonyskipper]

Your point is good, but that stack wouldn't win any latency awards. Many of the people I know using kafka need latencies in the millisecond range.

[KaiserPro]

but kafka isn't fast. Most things are backed by real files, so when you hit limits or something ejected from cache, it gets slow real fast.

Kafka isn't the right choice for most things.

SQS, MQTT, NATS, rabbit if you're wanting a lot of admin are all better (plus the crap that azure and google make)

[slt2021]

kafka is not for latency, it is used for high throughput. By design kafka shines in high throughput workload due to consumer and producer concurrency (consumer group), broker concurrency (multiple nodes and partitions).

for latency sensitive you will probably need redis pub/sub or something in-memory

[happymellon]

In all the times I've been forced to use Kafka, I have never seen single digit millisecond latency.

If you need fast response Kafka is a bad choice.

If you are okay going to multiple digits of milliseconds then there are simpler solutions.

The only reason to use Kafka is the ability to guarantee order. For everything else it's second place at best.

[pharmakom]

what if you want to have at least once processing and durability?

[happymellon]

Like AWS SQS? Others provide at least once processing as well.

Kafka has been the slowest out of them that I've used and definitely more complex to use.

[pharmakom]

SQS has durability for up to 14 days and you can only have one consumer group.

It’s also proprietary.

[happymellon]

Then use Rabbit.

It's still harder to get wrong than Kafka.