Hacker News new | ask | show | jobs
by diarrhea 1153 days ago
> Aren't all distributed databases basically really clever wrappers around write-ahead log + really tight timestamp/clock syncing?

As far as I know, the second requirement is often solved differently. Google’s Spanner has tight clock synchronisation via GPS and/or atomic clocks, and will even report uncertainties. Knowing these uncertainties allows it to simply wait them out before committing, for example.

But in general, exact time keeping and clock syncing is often too hard and costly. Luckily, it’s often not required and one can do with logical clocks, such as version vectors or Lamport time stamps. These order events by causality (A before B, B before A, A and B happened concurrently), which eventually allows the WAL to be sorted deterministically.

Things like multi leader with async replication will inevitably run into conflicts though. These will need some sort of resolution (manually or automatically via CRDTs). There’s no way around it due to the builtin, inherent possibility of concurrent writes.

Note that concurrent in these scenarios has essentially nothing to do with time. It’s not about “happened at the same time”. It’s a question of “did A know about B?”. No? Then A can’t be causally dependent on B and they are concurrent events. Exactly like two “parallel” branches in git. They’ll need to be merged later on, and conflicts will need to be resolved.

Lastly, if we can deterministically order events, every node can reach the same conclusions. This is equivalent to consensus.

So my take would be: distributed databases are often about a log of (write) events, and some consensus mechanism to agree upon the exact order in that log. Logical clocks are a good solution for that, but physical clocks ca be made to work as well (Google Spanner).

This is all taken from the book “Designing Data Intensive Applications”, a great read!