>If the leader ever dies, a new leader will be elected
>through a simple protocol that uses random sleeps and
>leader declarations.
Why not have each node self-generate a UUID and engage in some gossip process that ends with the cluster becoming aware that some node's corresponding UUID is uniquely significant, therefore recognizing that node as a leader?
I have some really bad memories of "random sleeps" at scale.
Well, from the sounds of it they're not running at scale (always 3 nodes), and it sounds similar in principal to the way HSRP/VRRP works which is a well defined and understood protocol for doing leader election on a local network.
I suppose the question might be why not use VRRP itself, but if this works for them and has conflict resolution I don't think it's all that troubling.
I have some really bad memories of "random sleeps" at scale.