[madmax96]

This one was kind of troubling, if you ask me:

    >If the leader ever dies, a new leader will be elected
    >through a simple protocol that uses random sleeps and
    >leader declarations.

Why not have each node self-generate a UUID and engage in some gossip process that ends with the cluster becoming aware that some node's corresponding UUID is uniquely significant, therefore recognizing that node as a leader?

I have some really bad memories of "random sleeps" at scale.

[kevin_nisbet]

Well, from the sounds of it they're not running at scale (always 3 nodes), and it sounds similar in principal to the way HSRP/VRRP works which is a well defined and understood protocol for doing leader election on a local network.

I suppose the question might be why not use VRRP itself, but if this works for them and has conflict resolution I don't think it's all that troubling.

[coffeesn0b]

Caleb with CFA here - random sleeps aren't exactly the most elegant solution... this was our MVP, and we'll definitely be improving on it.