[gshx]

Does someone have any benchmarks or comparisons with Zk? We have run it for many yrs without a problem and are very happy with it. Would be interested in hearing from anyone who switched from zk->etcd for distributed locking, presence, leader election type idioms and ran it in prod for a few months/yrs and their takeaways.

[ideal0227]

etcd3 has similar performance compared to ZK for small scale. For large dataset, etcd3 does better since it does incremental snapshot/ smaller memory footprint, when ZK does full snapshot that takes a lot of resources. For watches, etcd3 does streaming + TCP multiplexing to save memory.

[embiggen]

I too have been using ZK for many years now, and it's pretty great.

Etcd can provide faster election notifications but it comes at the cost of etcd still being pretty new, so be prepared to get cut by the bleeding edge :)

[Randgalt]

etcd is a key value store that has features of ZK. ZooKeeper is strictly distributed coordination. Apples and oranges, no?

[atombender]

Apples all the way. Etcd is pretty much a clone of ZooKeeper in Go; they both support hiearchical keys, atomic autoincrement, watches, though Etcd uses the Raft consensus algorithm, whereas ZooKeeper uses its own homegrown algorithm, and there are other minor differences. Both are intended for configuration management and coordination. (Of course, both are ultimately clones of Google's internal tool, Chubby.)

[embiggen]

I agree that they are similar in some ways, but under the covers they are fundamentally different beasts in almost all ways!!!

Etcd uses Raft and ZooKeeper uses it's own protocol called Zab [0]. Zab shares some characteristics with Paxos but certainly IS NOT Paxos.

[0] https://cwiki.apache.org/confluence/display/ZOOKEEPER/Zab+vs...

[lobster_johnson]

As I understand it, both Paxos and Zab will result in similar performance characteristics, since writes need, by design, to be coordinated with peers and serialized in a strict manner. In this sense, Etcd and ZK are very much alike, irrespective of how they are implemented internally. I wouldn't be surprised if Etcd was found to be faster and more scalable than ZK, however.

[ideal0227]

It really depends on how you view the problem. Yes, the latency of agreeing a proposal is similar, which is limited by physical (network latency + disk io). However, there are ways to put more stuff into one proposal (batching) and submitting proposals continuously (pipelining). These optimizations highly depend on the implementations.

[jonaf]

I believe ZK implements Pacis. Maybe someone not on mobile can correct me / provide reference.

[Randgalt]

ZK is zab - similar to Paxos but not Paxos

[gshx]

Your point is well-taken. This qn comes up often enough for similar use cases and hence I asked if someone has data to support an arg. Considering the criticality of Zk in the larger stack and our success and operational expertise with it, it has to take significant convincing to switch to something like etcd.

The following two paragraphs from the etcd project site seem to hint that they're trying to target overlapping use cases: "Your applications can read and write data into etcd. A simple use-case is to store database connection details or feature flags in etcd as key value pairs. These values can be watched, allowing your app to reconfigure itself when they change.

Advanced uses take advantage of the consistency guarantees to implement database leader elections or do distributed locking across a cluster of workers."