Hacker News new | ask | show | jobs
by seiji 4265 days ago
Picking out the three points: not highly available, not consistent, and not partition tolerant.

[reverting to non-standard, dummy definitions below just to explicitly define some things]

Redis Cluster is available as long as (50% + 1) of the master nodes are reachable by each other. So, 15 masters will be more "available" than 3 masters if they are deployed across a sane network topology. Your masters will have replicas as well, so if a single master instance fails, a replica for that master will be elected to be the new master.

Redis Cluster is consistent in that each replica is an exact copy of its attached master (by default, each master owns N/16384 of the keyspace, where N is the total number of masters). You can lose writes if the master doesn't have time to replicate before the master fails, but that's the nature of async replication. (You can request your Redis client "wait" for at least N replicas to receive your commands before the command completes; this gives you some assurances.) It's possible for you to get an "inconsistent read" if you read from an async replica after writing to the master, but you have perfect read-after-write consistency if only talk to master instances.

Redis Cluster nodes on the minority side of a partition will, by default, deny reads and writes until they re-join the cluster. Redis Cluster prefers data integrity over availability by default since Redis Cluster has no merge operations. You can optionally allow the minority side to keep accepting commands, but that definitely will not guarantee any consistency of your data when the whole cluster reappears.

Given that better AP systems exist, ones that actually provide things like availability during partitions and eventual consistency mechanisms/guarantees, will Redis Cluster eventually support these features?

The one limitation for each of those features is: Redis is an in-memory database. Riak uses multiple KB of metadata for each object in the DB. With Redis, it's common to have tens of millions (or hundreds of millions) of keys on one server.

Availability during partitions with eventual consistency (if you want more than last-write-wins) requires CRDT-like things, which requires metadata, which requires more memory usage per-key.

Full and usable availability during partitions is nice to think about, and it would be great to one day have the option to maybe have in-memory Redis "better data consistency for smaller, manageable datasets" with CRDT merge operations so you get a Reis-dynamo type thing.

But, since Redis Cluster is already 4 years under development, it's not worth burning brain cycles on until real users start using Redis Cluster and we can see where the needs-vs-features plot falls. (plus, the issue backlog for other Redis development improvements is about 100 tasks long, so speculative improvements have to fall in line with the balance of Cluster vs. Standalone vs. Master-Replica vs. New Commands vs. Improving Existing Commands vs. Bug Fixes vs. User Contributions vs. Doc Updates vs. Evangelism vs. ...).