[jorangreef]

Haha! You could double down and up the stakes.

It’s not that it would need dynamic memory allocation (it could be done with static), but rather it’s not essential to performance—you could use any KV or OLGP for additional “user data”, it’s not the hard contended part.

To keep consistency, the principle is:

- write your data dependencies if any, then “write to TB last as your system of record”,

- “read from TB first”, and if it’s there your data dependencies will be too.

[testdelacc1]

That makes sense, but it seems to add latency? Let’s say we’re currently reading the transactions from Cassandra with latency t. If TB is the source of truth, but we need an additional read from Cassandra for every transaction, the read latency is strictly worse now. Similarly with writes, especially since you recommend writing to TB last.

If what I’m saying is correct, we won’t actually see any performance benefits, only possible regressions. And if we’re already happy with Cassandra’s record keeping, what does TB add here?

Correct me if I’m missing something though! Like maybe we could rework how we write to Cassandra.

[jorangreef]

Fair question!

If you have any contention in your workload, which is typical for OLTP workloads [1], then, per Amdahl’s Law, that portion of the work will dominate, and TB will be orders faster.

For example, if your KV can do high throughput writes (and most can) at low latency then you should be looking at 20-50ms P100s all combined, per 8K transactions, when you’re running at 500K TPS, even with extreme contention up to 90%. Very hard if not impossible to do that if you don’t have TB as part of your stack. At least not with the durability and strict serializability guarantees that TB gives.

[1] Worth watching! https://youtu.be/yKgfk8lTQuE