Hacker News new | ask | show | jobs
by vorpalhex 2023 days ago
That's a protocol problem, use a protocol such as rsync. You don't need to use redacted sends/recvs.
3 comments

rsync doesn't scale like zfs send/recv. It requires scanning of every file at both the source and destination to compute the delta to send. zfs snapshots and send/recv don't need to do that. The delta is already fully described by the snapshots themselves. zfs is also working with immutable snapshots. It guarantees the source and destination copies are identical; rsync can't do much about the source and destination being modified while it is running since it's reliant upon other users of the system not touching the data being synced.

That's not to say rsync doesn't work. It does. But it doesn't scale well, and the data integrity guarantees aren't there.

rsync has it's own issues if the connection has high latency though - zfs send was originally developed by a Sun engineer who wanted to speed up large transfers to servers in China, if I recall correctly.
+1 for rsync, but with check-summing turned on, i think that's acceptable for 40GB.
It's not really enough for ZFS (unfortunately). It won't move snapshots, bookmarks etc.