The video lists pros/cons for strongly consistent and eventually consistent databases, but only has pros for a "just right consistency" database. What are the cons?
The idea of the "just right consistency" is that it brings the best of both worlds, without any drawbacks.
Your application works as well as if it was executed fully in strong consistency, but with improved scalability for the set of operations that can execute in an eventual consistent model.
The biggest drawback is that only some operations can be supported. E.g., without strong consistency you can detect double-spending from an account but you can't prevent it, because the validity of an operation can't depend on operations a datacenter hasn't seen.
Financial examples are bad because in fact the financial world IS eventually consistent. It's quite possible to withdraw the same $100 from an account via multiple ATM machines.
With ATMs and debit cards, I thinks it's generally not true, they seem to use the online mode and update the balance of a checking account within seconds.
With credit cards, you can indeed start more transactions against the same balance, and you're never sure in which order they will complete.
When a payment processing system doesn't promptly cancel auth holds, customers definitely complain about being prevented from spending their money. This stuff should be table stakes but some retail banks are just way behind.
That is the reverse problem. Flippin' gas stations STILL have a problem with this that you feel acutely because the large holds. A similar problem exists with hotels, where they'll put a lock on a ton of money in your account.
Great video. I'm impressed you guys built the checker tool, that isn't an easy task - we built something similar called PANIC (https://github.com/gundb/panic-server) that lets us simulate failure cases and then run the test across a distributed system to see if it passes.
I had a question, in the video at a certain point you say that you must modify the code to disallow concurrent debits. This makes sense in theory, but wouldn't it fail in practice? If two machines in different regions are running this code, they would not know that there is a concurrent debit. How si that addressed?
Correct database implementations must be strongly consistent, unless your updates all commute with each other, in which case eventual consistency works.
E.g. db[key] += value and db[key].insert(value) commute with themselves, but db[key] = value obviously doesn't.
This just seems to be an attempt to implement a correct eventually consistent database, and the CRDTs are simply datatypes with commutative update operations.
Unfortunately, it seems to allow transactions that examine objects and then (conditionally) update them, which obviously are not guaranteed to be commutative even if the objects are commutative, so maybe it doesn't succeed at implementing a correct database.
Some conditional updates are safe; others require to add concurrency control. Our CISE analyser will tell precisely you which side a specific operation falls into. See https://youtu.be/HJjWqNDh-GA and http://dx.doi.org/10.1145/2911151.2911160. Antidote doesn't yet support concurrency control, it's work in progress.
Your application works as well as if it was executed fully in strong consistency, but with improved scalability for the set of operations that can execute in an eventual consistent model.
https://www.youtube.com/watch?v=HJjWqNDh-GA