Hacker News new | ask | show | jobs
by tsuraan 1542 days ago
Speaking of snapshots, are you planning to have analogs to ZFS's send/recv and diff? I didn't see any mention of them on the snapshots page or the roadmap, but I could be looking in the wrong place. Already having reflink support is exciting though; a ZFS replacement that can also do reflinks would be amazing.
1 comments
koverstreet 1542 days ago
Yeah but it's not really designed yet. We have the main pieces - all metadata exists as keys in various btrees, and keys all have version numbers so we can easily do a protocol to transfer all keys newer than x - and then your remote filesystem will be up to date.

To make it efficient and fast we'll want to be able to do it without scanning all filesystem metadata, which will require auxiliary indices, and that'll probably need some careful design to make it fast. But we've got transaction layer triggers, so we've got the model for that stuff in place.

Also reflink support is plenty solid at this point. I can't really recommend snapshots for production use yet - in particular, we're missing per snapshot/subvolume disk space accounting, which you kind of need. But reflink's been in use and getting tested for ~2 years now, and there's no known issues with it anymore.

link
tsuraan 1541 days ago
Okay, good to know it's being considered. For a non-optimized diff then (zfs-diff, which just prints which files have changed, not what about them have changed), it would be the same approach? Scan all the metadata looking for keys with versions between the two snapshots, and print out the associated file names? Would that simple approach be likely to be faster than whatever rsync does to find updated files when doing a backup?
link
koverstreet 1541 days ago
If you just want to know what files have changed, there's no reason to scan all metadata - you'd just want to scan all the inodes (and we'd need to add a version number field to the inode itself that covers all the contents; easily done). And since just prior to snapshots inodes have backpointers back to dirents, so given an inode we can easily get a path.

So yeah, that would be easy and a lot faster than what rsync does. Dunno about plumbing to anything in userspace, that's outside the scope of what I can think about right now. But if anyone wanted to work on such a feature, I'd happily help them get started.

link