[hnkimb3558]

Performance is a very tricky thing to measure in a database. CockroachDB's performance is certainly affected by its consistency model. In particular, CockroachDB handles transactions using serializable isolation, and writes using consensus replication. This means that it's difficult to make an apples to apples comparison for performance between a replicated CockroachDB cluster and a single master MySQL server. However, that is the whole point of a well thought out, standardized benchmark like TPC-C. It creates a pseudo-realistic workload, and mandates the use of transactions with a reasonable isolation level or higher (snapshot isolation), carefully calibrated degrees of contention and failures, and then leaves it up to the database vendor to optimize within those parameters.

When Amazon did this for Aurora, they showed that with their custom storage backplane, they were able to absolutely murder normal RDS-based MySQL in terms of TPC-C throughput. Take a look: https://www.slideshare.net/AmazonWebServices/dat202getting-s.... The bottom right quadrant of that slide shows what happens to a standard MySQL server when given 10,000 TPC-C warehouses to chew on (about 800 GB of TPC-C dataset). The max throughput is supposed to be ~128,000 tpmC. RDS MySQL in their tests was able to do 69. That is quite simply, not impressive. Even Aurora, which is a full 136x the tpmC throughput, is still more than 10x less than the max throughput that should be achieved at that number of warehouses.

CockroachDB can scale out, so while it may have more latency when it performs a simple transaction or millions of simple transactions over a small set of data, it can also easily scale to handle 10,000 TPC-C warehouses, with 126,000 tpmC. If you're going to talk about performance, you really need a serious benchmark, or you're just kidding yourself.

Let me put this another way. While Amazon frequently talks about how Aurora can be scaled to 64TB of data, in the context of TPC-C, that's a risible claim. A maximal instance of Aurora, tuned by AWS, can handle max TPC-C throughput at something between 80GB and 800GB of data (i.e. 1,000 to 10,000 warehouses). It's like they're selling you a pickup truck with a bed they say can handle 64 tons, only if you load it that way, it can only travel 2mph, and can't turn. Caveat emptor!

[amq]

When I tested CockroachDB, I did a single-node and multi-node comparison to Galera. I was ready to accept that multi-node is going to have a performance hit, but what surprised me the most was that the single-node performance was roughly 5-10 times worse with CockroachDB.

https://github.com/cockroachdb/cockroach/issues/17777#issuec...

[hnkimb3558]

I just read up on that issue. We will follow up with updated measurements with CRDB 2.0. The big reason for the discrepancies you measured is likely the fact that each sqlbench write transaction is doing six consecutive consensus writes (incl. commit). With CRDB, you need to parallelize your transaction writes using the `RETURNING NOTHING` syntax, which would likely be a significant improvement. This manual tuning is obviously sub-optimal, and will not be required in the next version of CockroachDB (2.1 in October).