| You're right, my comment would have been more valuable if I'd included some alternate technologies they could have used instead, but I believe that the criticisms still stand regardless, because the approach itself is in question. You don't find out once you've reached scale that the underpinnings for your distributed KV platform are not actually free of global locks for writes! That's something you verify and test for yourself before building your empire upon it. If an alternative exists, you use it. If not, you need to build it. To answer your question: > what would you have picked back in _2011_ when Cloudflare was getting off the ground? Not only "would have" but did pick and use (well before 2011) an abstraction around SQLite because our team first evaluated our read vs write requirements and found it to be an adequate option rather than going crazy trying to find a nosql solution worthy of including on our resumes. The Cloudflare article is somewhat skimpy on the details of their benchmark, so these numbers are not an exact equivalent but, here, I just threw this together: https://github.com/mqudsi/sqlite-readers-writers P99.9 for reads with two writers is 2ms as compared to their 1215ms, and this is with full ACID compliance and write synchronization. You don't need to be an expert in creating these systems, you just have to be able to test and validate your architectural decisions before building upon them. It takes only an hour or three to pick a library and write a similar benchmark for any KV store you're interested in (although the exact benchmark would have to be tweaked to match your expected needs). (That said, lmdb is a great choice and I've recommended it here on HN before... except they ended up replicating on top of it a lot of what SQLite provides for free. Their transaction logs are almost like SQLite's WAL which I used in my benchmark, except going with SQLite would have skipped the entire second half of their Quicksilver solution since it they wouldn't need to manually handle transaction logs, split payloads, and manually reassemble to avoid fragmentation.) |