[manigandham]

Looks interesting, might be a good replacement for the more enterprise messages/queue systems that have all the typical ack/redelivery/scheduling features as seen here.

It's worth mentioning the new Apache Pulsar messaging system which can replace Kafka with pub/sub and queueing semantics while providing better scalability and per-message acks, probably better suited to those who want a combined system.

[arcbyte]

I checked out Pulsar and got completely lost in the multiple hierarchical Zookeeper clusters.

[manigandham]

Pulsar supports multiple regions natively which requires separate Zookeeper clusters for each region to manage the global and local cluster state (ie: replicating messages from DC1 => DC2 but not DC3).

If you dont need/want that, then its just a single ZK cluster as with Kafka or anything else. ZK + Brokers + Bookies = Pulsar.

[aphistic]

I hadn't heard of Pulsar before but just checked it out as we use Kafka. Does it support replaying messages from an offset (or even better, a time) like Kafka does? I looked at the docs but it didn't seem like it did.

[danellis]

So many Apache projects, and with a lot of overlap, too. It would be nice if someone made some kind of infographic that gives a high level overview of them all.

[pquerna]

not quite an infographic, but this site lets you slice the projects by different attributes:

https://projects.apache.org/