[nuopnu]

The presentation and the paper pretty much describe how LMDB works already.

https://symas.com/lightning-memory-mapped-database/

Except, it allows only a single writer and it's not directly byte-addressable, unless you make a request to preallocate a chunk of fixed-size mmap'ed memory.

[jarulraj]

LMDB is a highly-optimized storage engine. However, it is not a full-featured DBMS (e.g., no support for query optimization). This talk focuses on the potential impact of NVM on different layers of a full-featured DBMS -- not only the storage engine.

[valarauca1]

Yes but in byte-addressable mode you can trust the map is fixed, which doesn't seem to be the case here.

[nuopnu]

Not exactly. I'm not a user of this, but

http://www.lmdb.tech/doc/group__mdb__env.html#ga492952277c48...

Ctrl+f through the doc for "MDB_FIXEDMAP" for more details.

So there is _some_ support, where you can expect to have the whole file at a fixed address. What is not handled is that your data may be moved within the file, and there's nothing you can do about it, short of keeping a long-lived read only transaction:

http://www.lmdb.tech/doc/todo.html

[hyc_symas]

You have it right.

In practice, nobody has wanted support for this so it has remained unimplemented. Note that if your data items are large enough (more than 1/2 a page) they'll go into their own overflow pages, and then won't shift around at all. (But if you delete/modify a value, the new version will of course be in a new overflow page.)

Granted, this could be a case of "if you build it they will come" - before LMDB existed, nobody cared about mmap'd transactional DBs either. And personally, I still see good use cases for it.

[cryptonector]

Note that the LMDB approach, and especially if you want MDB_FIXEDMAP, limits DB size to the largest mmap()ing you can get at a fixed location. That's not good in a world of 48-bit address spaces.

[nuopnu]

Practically it's even less than 47 bits. :) But sure, know your limits.

[cryptonector]

Yes, I know :)