[elvinyung]

(Disclaimer: I work at Square, but not on Books.)

>Preventing inconsistencies by enforcing constraints is a key point of an RDBMS. This is literally why people use SQL DBs.

This comment is not specific or related to the work at hand, but note that this is only true for a single-machine database or a sharded setup in which you never have to perform transactions across shards. That is, the ACID guarantees are only enforceable in most RDBMS if all your data is on the same machine.

[josephg]

Huh? Am I missing something obvious here? Plenty of databases support ACID guarantees across multiple machines in a cluster. FoundationDB and CockroachDB come to mind, but I’m sure there are others. What you’ve built is cool, but I’m still confused why existing tools aren’t good enough.

From the Cockroach documentation (source: https://www.cockroachlabs.com/docs/stable/transactions.html )

> CockroachDB supports bundling multiple SQL statements into a single all-or-nothing transaction. Each transaction guarantees ACID semantics spanning arbitrary tables and rows, even when data is distributed.

[elvinyung]

Right -- that's why I said most RDBMS. To get ACID and CAP-consistent (which, as an aside, are not the same thing) cross-shard transactions requires some form of migrating your production system onto such a distributed-capable RDBMS that you describe (of which, Spanner arguably fits the description of).

[josephg]

The thing I'm confused about is why migrating your production system onto a custom in-house data store (that you've written and you maintain) is easier / less work than migrating onto something you can use off the shelf which solves the problem you have.

I hear the argument that migrating from (say), postgres -> postgres + books is easier than moving everything into cockroachdb. But why is postgres + books easier than postgres + cockroach? The latter doesn't require you to write your own database from scratch. (And writing your own database is fun, so if you're doing it for that reason I understand, but thats not what the post says.)

Edit: Ah, I see - my confusion was because I missed the part which mentioned that this is just built on top of spanner. That makes sense!

[elvinyung]

Yep :P

For a fun fact, CockroachDB was actually started by ex-Squares, so we're definitely very familiar with this exact same argument internally, back in the day when neither CockroachDB nor Spanner existed as a viable option :)

[justinjlynn]

Not everyone has the luxury of using databases like those in high-stakes production.