[cmrdporcupine]

Obligatory: https://db.cs.cmu.edu/mmap-cidr2022/

Consensus says "don't do it" ...

That said, having written my own buffer pool and paging, etc... in pure naive benchmarks ... it's actually kinda hard to beat mmap. And LMDB is really fast for what it is.

In real world workflows I think the story is more complicated. Especially under higher concurrency.

[hyc_symas]

Obligatory "that paper is garbage" https://www.symas.com/post/are-you-sure-you-want-to-use-mmap...

[quotemstr]

I read the linked post. You're not making a good argument.

The authors aren't arguing that a mmap database is worse because it's "more complex". They are arguing it must work with less information. You haven't refuted the original paper, but you have made me more skeptical of LMDB.

For example, you claim that applications "never" have control of memory. That's simply, again, false. We have explicit memory eviction and pinning operations. We even have VA-batched TLB shootdown IPIs via process_madvise. On some systems (AMD, soon Intel) we can do TLB invalidation without an IPI.

So no, you're just wrong in making the claim that you might as well use mmap because you can't control the memory lifecycle anyway. You absolutely can, and anyone reading this message can look up the relevant APIs for himself.

And you point to LMDB's benchmarks repeatedly as evidence you're right. That's not saying what you think it is. LMDB is fast despite being hobbled by vanilla kernel mmap. Yes, that means other databases are probably doing stupid things, but reverse stupidity is not intelligence.

[hyc_symas]

You're dreaming. None of your explicit memory control operations mean anything in practice, because today everything runs in VMs with no actual control of the underlying hardware. Probably co-resident with an unknown number of other tenants.

As for what you claim the paper's authors were saying - I quoted their text verbatim. Your interpretation is not what they said.

They claimed using mmap safely is impossible, and using it correctly requires more complexity than a traditional DB design. The safety claim was already disproven by multiple researchers. To prove their second claim they would have had to produce a DB that did traditional buffer management and was simpler and more performant than using mmap. They never did any such thing, nor could they.

[radiator]

"we don't have to pay back any vulture capitalists"! a good one