[g0del_was_wr0ng]

Including your 9x write amplification in the number of "consistent writes" doesn't count -- like at all. I'm amazed nobody called you out on this yet.

You're doing 3k batches per second with 4 logical writes each, right? So that is at most 3-12k writes per second using the way that every other distributed database benchmark and paper counts.

Or otherwise - if you continue counting writes in this special/misleading way - you'd have to multiply every other distributed db benchmark's performance numbers with a factor of 3-15x to get an apples-to-apples comparison.

The 12k batched writes/sek through what I assume is a paxos variant is still pretty impressive though! Good to get more competition/alternatives for zookeeper & friends!

[evanweaver]

No, that's not write amplification. Replication and storage engine fanout are not included. Instead, that number is the number of logical partition updates per row, per transaction. This makes the test comparable to tests of key-value stores that can only update one key per transaction. The FoundationDB test mentioned elsewhere here was reported the same way.

If you want to include write amplification, then multiply by 6x again to account for the replicated log and the tables themselves.

[g0del_was_wr0ng]

It's doing 12k rows in 3k user-issued write operations/transaction per second.

Counting any kind of "internal write effects" that result from a user write (i.e. write amplification) is obviously done to mislead in the benchmark and does not make it comparable to key-value stores.

12k writes/s is the number of rows that are written from a user perspective. So 12k/s is also the number you have to use when comparing it to key value stores. But of course, comparing Fauna with eventually consistent systems is not a really fair comparison. You don't make it fairer by misleading in your benchmark though.

Also, just because some other vendor posted a misleading benchmark on hn (I don't know if they did) that doesn't make it right or means you should do it. Just call them out on it too.

[evanweaver]

Indexes aren't internal write effects, they are user-defined. But we will have additional benchmarks later on that focus on row commits only.

We tried to replicate a realistic workload rather than just target the best case or worst case performance profile.

[g0del_was_wr0ng]

By that logic a write to a postgres table would also count as N+1 writes, where N is the number of secondary indexes defined on that table?

And also, by the same logic, replicating a write to three machines counts as three writes when the replication factor is user-defined, right?

[evanweaver]

For purposes of this benchmark, the first yes, the second no. But neither of them are free.

[g0del_was_wr0ng]

Well fair enough but that method of counting is not what everyone else does or assumes, so somebody just reading the title "120k writes per second" gets the wrong impression of whats going on.

(An uninitiated reader would assume you're comitting 120k rows per second from the title, whereas it's "only" 12k rows and "only" 3k actual operations over the wire. Still, 3-12k is pretty impressive)

[rystsov]

Even if we focus on 120k writes and ignore the low transaction number then it's still an average result. They use c3.4xlarge to run 5 shards each with 3 replication factor. It's 1500 writes per core (120k / (5*16)). Etcd easily does 5227 writes (I believe it can do even more) on "Standard DS12 v2" so it's 1306 per core which is very similar.

[eternalban]

> nobody called you out on this yet.

https://news.ycombinator.com/item?id=13728738