[evanweaver]

No, that's not write amplification. Replication and storage engine fanout are not included. Instead, that number is the number of logical partition updates per row, per transaction. This makes the test comparable to tests of key-value stores that can only update one key per transaction. The FoundationDB test mentioned elsewhere here was reported the same way.

If you want to include write amplification, then multiply by 6x again to account for the replicated log and the tables themselves.

[g0del_was_wr0ng]

It's doing 12k rows in 3k user-issued write operations/transaction per second.

Counting any kind of "internal write effects" that result from a user write (i.e. write amplification) is obviously done to mislead in the benchmark and does not make it comparable to key-value stores.

12k writes/s is the number of rows that are written from a user perspective. So 12k/s is also the number you have to use when comparing it to key value stores. But of course, comparing Fauna with eventually consistent systems is not a really fair comparison. You don't make it fairer by misleading in your benchmark though.

Also, just because some other vendor posted a misleading benchmark on hn (I don't know if they did) that doesn't make it right or means you should do it. Just call them out on it too.

[evanweaver]

Indexes aren't internal write effects, they are user-defined. But we will have additional benchmarks later on that focus on row commits only.

We tried to replicate a realistic workload rather than just target the best case or worst case performance profile.

[g0del_was_wr0ng]

By that logic a write to a postgres table would also count as N+1 writes, where N is the number of secondary indexes defined on that table?

And also, by the same logic, replicating a write to three machines counts as three writes when the replication factor is user-defined, right?

[evanweaver]

For purposes of this benchmark, the first yes, the second no. But neither of them are free.

[g0del_was_wr0ng]

Well fair enough but that method of counting is not what everyone else does or assumes, so somebody just reading the title "120k writes per second" gets the wrong impression of whats going on.

(An uninitiated reader would assume you're comitting 120k rows per second from the title, whereas it's "only" 12k rows and "only" 3k actual operations over the wire. Still, 3-12k is pretty impressive)