[rdtsc]

To summarize quickly for those that didn't read the article, this focuses mostly on the new "clustering" aspect not on the "classic" single Redis server (if you wish).

I think it is important to be honest with the users and make it clear how and what happens behind the scenes, how data could be lost. And Salvatore has done most of this, maybe just make it a bit more explicit, as there still seems to be some confusion around.

All this is in light of 2 things -- 1) With the popularity and amount of talks and churn around distributed systems these days, people sort of expect a point on the map in the CAP triangle. So just saying we kind of do this and we provide some C, a little A and a dash HA was probably ok 5 years ago, now it needs a bit more definition, 2) In light of other database systems misleading users about what it could provides (you know which one I am talking about) and having resulted in lost data, there is a bit of apprehension and a higher bar that needs to be met in order for a db product to be accepted.

One good thing that came out recently is NoSQL database writers/vendors pushing for more rigorous tests. Tests that run for weeks and months. Consistency tests, network partition tests as run by Aphyr. It is a very good idea those things are talked about and defined better.

[fizx]

I think Antirez is simply doing a master/slave system with async replication. This is what mysql, solr, elasticsearch, kafka, etc do. This is not an unusual model at all.

In CAP theorem terms, Redis has picked zero (remember CAP theorem is pick at most two).

There's a bunch of people who've made this choice, but why? C incurs a synchronization cost. A means that you have to reconcile different writestreams. If you want consistent semantics in a very fast database, you can't pick either. So you end up somewhere in the middle of the triangle.

The consequence of picking zero is that you'll lose a time window of data roughly proportional to the replication lag when the master fails/partitions. There are many applications for which bounded data loss is a perfectly reasonable paradigm.

[aphyr]

I assert that this design actually allows for arbitrarily long windows of data loss, but I haven't verified that the implementation matches my understanding of antirez's WAIT/failover algorithm yet. Pretty sure though.

[antirez]

If you read the Redis Cluster documentation, the data loss is acknowledged, even explaining what are the most obvious failure modes where this happens. WAIT does not provide strong consistency either, and is actually not even documented in the Cluster doc. However WAIT as it is makes the possibility for data loss less likely.

Once everything in the Redis Cluster design and documentation is about trading consistency for performance, I would expect the system to be analyzed for what it is: is it good at providing weak but practically usable consistency and performances? In short, it does respect the design intent?

Saying again and again that it features not consistency is a sterile exercise.

[pfraze]

The strongest guarantee needs to be how quickly the system catches failure to recover or halt.

[chubot]

In CAP theorem terms, Redis has picked zero (remember CAP theorem is pick at most two).

That might be true, but describing something in the CAP framework is not the only salient thing you can say about a distributed storage system. It is characterizing the failure modes.

You also have to think about what happens in the normal case. In the normal case you have a consistency vs. latency tradeoff. People have written about this, but unfortunately I don't think this broader way of thinking hasn't the attention it deserves:

http://dbmsmusings.blogspot.com/2011/12/replication-and-late...

http://dbmsmusings.blogspot.com/2010/04/problems-with-cap-an...

"If you examine PNUTS through the lens of CAP, it would seem that the designers have no idea what they are doing (I assure you this is not the case). Rather than giving up just one of consistency or availability, the system gives up both!"

"The reason is that CAP is missing a very important letter: L. PNUTS gives up consistency not for the goal of improving availability. Instead, it is to lower latency. "

[rdtsc]

I was just making a point since even people who have a better average understanding of distributed systems, can't quite put their finger on what Redis Cluster's failure modes or guarantees are, perhaps a bit more of clarification is needed.

Even you are using "I think antirez is simply doing..."

Not arguing the design is not useful or it is a bad design, Savlatore's code is great, I think it is just a matter of more testing, docs or a blog post.

[skrebbel]

> In CAP theorem terms, Redis has picked zero (remember CAP theorem is pick at most two).

No, he picked P, Partition Tolerance. You can't not pick P. Unless you assume a never-failing network with never-failing nodes.