Hacker News new | ask | show | jobs
by formerly_proven 980 days ago
There's actually an NVMe command set which allows you to use the FTL directly as a K/V store. (This is limited to 16-byte keys [1] however, so it is not that useful and probably not implemented anywhere, my guess is Samsung looked at this for some hyperscaler, whipped up a prototype in their customer-specific firmware and the benefits were lesser than expected so it's dead now)

[1] These slides claim up to 32 bytes, which would be a practically useful length: https://www.snia.org/sites/default/files/ESF/Key-Value-Stora... but the current revision of the standard only permits two 64-bit words as the key ("The maximum KV key size is 16 bytes"): https://nvmexpress.org/wp-content/uploads/NVM-Express-Key-Va...
3 comments
londons_explore 980 days ago
Presumably there is some way to use the hash of the actual key as the key, and then store both key and value as data?

16 bytes is long enough that collisions will be super rare, and while you obviously need to write code to support that case, it should have no performance impact.

link
formerly_proven 980 days ago
A 32-byte key would allow using NVMe KV directly for content-addressed storage; many of those systems use 256-bit / 32-byte cryptographic hashes as keys. Notable exception would be git with 20-byte keys.
link
londons_explore 980 days ago
You can still do this... Just use the first 16 bytes as the key, and the 2nd 16 bytes as the start of the data.
link
londons_explore 980 days ago
I think some devices built the block storage on top of the key-value store. Ie. when you write "hello..." (4k bytes) to address 123, it actually saves key: 123, value "hello...".

If so, that is probably the reason for a 16 byte key - there is just no way anybody needs a key bigger than 16 bytes for an address anytime soon.

link
londons_explore 980 days ago
I could imagine that if this mode isn't widely used, drive manufacturers haven't given much thought to performance, and it therefore might suck.
link