Hacker News new | ask | show | jobs
by magicalhippo 1082 days ago
Internally, ZFS is kinda like an object store[1], and there was a project trying to expose the ZFS internals through an object store API rather than through a filesystem API.

Sadly I can't seem to find the presentation or recall the name of the project.

On the other hand, looking at for example RocksDB[2]:

File system operations are not atomic, and are susceptible to inconsistencies in the event of system failure. Even with journaling turned on, file systems do not guarantee consistency on unclean restart. POSIX file system does not support atomic batching of operations either. Hence, it is not possible to rely on metadata embedded in RocksDB datastore files to reconstruct the last consistent state of the RocksDB on restart. RocksDB has a built-in mechanism to overcome these limitations of POSIX file system [...]

ZFS does provide atomic operations internally[1], so if exposed it seems something like RocksDB could take advantage of that and forego all the complexity mentioned above.

How much that would help I don't know though, but seems potentially interesting at first glance.

[1]: https://youtu.be/MsY-BafQgj4?t=442

[2]: https://github.com/facebook/rocksdb/wiki/MANIFEST