Hacker News new | ask | show | jobs
by diggan 980 days ago
I don't remember exactly why I have any of them saved, but these are some experimental data stores that seems to be fitting what you're looking for somewhat:

- https://github.com/DataManagementLab/ScaleStore - "A Fast and Cost-Efficient Storage Engine using DRAM, NVMe, and RDMA"

- https://github.com/unum-cloud/udisk (https://github.com/unum-cloud/ustore) - "The fastest ACID-transactional persisted Key-Value store designed for NVMe block-devices with GPU-acceleration and SPDK to bypass the Linux kernel."

- https://github.com/capsuleman/ssd-nvme-database - "Columnar database on SSD NVMe"

4 comments

See https://www.snia.org/educational-library/key-value-standardi... for some description of the special command set to get an nvme drive to natively work as a key-value store.

Also https://www.snia.org/sites/default/files/ESF/Key-Value-Stora...

How do you tell which NVMe drive models support the KV API? Is this something that you can experiment with on a consumer drive or do you need specific enterprise ssd models?

Samsung's uNVMe evaluation guide (from 2019) device support section just states:

    Guide Version: uNVMe2.0 SDK Evaluation Guide ver 1.2
    Supported Product(s): NVMe SSD (Block/KV)
    Interface(s): NVMe 1.2
https://github.com/OpenMPDK/uNVMe/blob/master/doc/uNVMe2.0_S...

I can't find detailed spec sheets detailing which NVMe command sets are supported even for their enterprise drives.

I reached out to Samsung support to ask. After being sent from one department to another and receiving some very clearly incorrect advice from their sales support they eventually sent me to an online form for the memory department. Am still waiting for a response a week later.
Interesting, never heard of this before! Do you have any other resources to share? How can I play with this today?
How is that implemented? Btree, hashtable?
Implementation-defined. The API resembles a map access, though.
RonDB is open-source and supports on-disk data on NVMe disks. http://mikaelronstrom.blogspot.com/2022/04/variable-sized-di...
Hey, thanks for the mention! UDisk, however, hasn't been open-sourced yet. Still considering it :)
you could also configure Redis to transact everything to disk and choose nvme as the target
That would save via file system, not bypass the kernel to access the NVMe drive directly from user space. NVMe drives themself have a bunch of features that make them amenable to K/V storage directly.

Good overview: https://www.mydistributed.systems/2020/07/towards-building-h...