[Plugawy]

The document mentions using Raft for consensus and coordination - it's the same approach used by Etcd, Consul, Serf, RethinkDB and other systems. AFAIK it's easier to implement and understand than Zookeeper's consensus solution.

https://raft.github.io

[ypcx]

Kafka was thinking about etcd:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-273+-+...

Considering most Kafkas will probably run in Kubernetes at some point, they could have shared the etcd used by Kubernetes.

[edude03]

There is also Zetcd[0] if you want to use etcd and kafka together today. We use it right now and it seems to work

[0] https://coreos.com/blog/introducing-zetcd

[Plugawy]

Sounds like a bad idea, no? Last ting you want is whole cluster going down because Kafka is misbehaving

[ypcx]

I don't know. First, there may be some separation achievable - but I'm just guessing. Second, if Kafka is your primary workload, you don't want it to misbehave in any case. But of course I get your point. I'm just saying I've seen people thinking about it that way in some Github Issues.