[taf2]

I wish redis had synchronous write option to guarantee that when a write response is received that all nodes in the cluster have also received a copy. Even if this was just an ack that data is replicated into memory instead of disk. Sure there are cases where you'd rather operate with async, but really I would prefer to put haproxy in front of 3 - 6 redis nodes and let it load balance both reads and writes similar to galera cluster with keepalived and a vip I would not need to be concerned with split brain assuming synchronous writes...

[seiji]

That's in the plan, it's just not implemented yet. (It's implied under the Note: Redis Cluster in the future will allow users to perform synchronous writes when absolutely needed.)

Something like:

   SYNC 3
   SET key value
   [replies with success only after it's ACK'd on three cluster instances.]

[taf2]

I would definitely take slower write speed to have this extra assurance of high availability with data consistency... as it is today - it's pretty hard operate redis in a HA way even with sentinel... I'm sure others have figured it out but for me the best I have is a 30 second window when sentinel is switching the backup slave to master and either writes are lost haproxy can hold but it works sometimes... I can imagine with synchronous replication we'd have slower write speed but at least failover could be sub-second and possibly zero downtime... using a vip/keepalived - Still redis is such a good datastore, it's difficult to imagine a world/service without it. for what it's worth this looks to be the best setup i've found: http://failshell.io/sensu/high-availability-sensu/ would be interested if others have other setups that possibly work better...

[antirez]

Hey, going a bit off topic about Sentinel, this should not happen. Sentinel + Redis + Clients as a distributed system is not able to guarantee strong consistency with arbitrary partitions clearly, but in the scenario you describe, that is, the master crashes or is partitioned away alone, the window should be like a fraction of a millisecond (the replication link latency) most of the times.

EDIT: to clarify:

1) Sentinels will tell clients master is A.

2) A fails.

3) Sentinels will still tell it is A before failover.

4) Failover happens.

5) Sentinels will finally tell master is B.

[taf2]

could be something that haproxy is causing - when switching masters... maybe I need a script that handles switching the vip when sentinel detects failure in master instead of haproxy with a backup... btw - thank you for redis.

[seiji]

You shouldn't need haproxy in the loop at all.

If your redis client is Sentinel-aware, you ask your client directly "give me the redis master server" and it asks Sentinel which server is master. The only ip address involved for your configuration is the address of the sentinel(s).

haproxy will take 30 seconds to detect your failure. Sentiel will notify you immediately. Sentinel is the one determining the exact state of the master election, and it'll be very chatty when a new master replaces the old one.