Hacker News new | ask | show | jobs
by MayeulC 266 days ago
I am quite confused; doesn't rsync already use content-defined chunk boundaries, with a condition on the rolling hash to define boundaries?

https://en.wikipedia.org/wiki/Rolling_hash#Content-based_sli...

https://en.wikipedia.org/wiki/Rolling_hash#Content-based_sli...

The speed improvements over rsync seem related to a more efficient rolling hash algorithm, and possibly by using native windows executables instead of cygwin (windows file systems are notoriously slow, maybe that plays a role here).

Or am I missing something?

In any case, the performance boost is interesting. Glad the source was opened, and I hope it finds its way into rsync.

3 comments

> doesn't rsync already use content-defined chunk boundaries, with a condition on the rolling hash to define boundaries?

No, it operates on fixed size blocks over the destination file. However, by using a rolling hash, it can detect those blocks at any offset within the source file to avoid re-transferring them.

https://rsync.samba.org/tech_report/node2.html

The readme very nicely contrasts the approach with rsync.
rsync seems frozen in time; it’s been around for ages and there are so many basic and small quality of life improvements that could have been made that haven’t been. I have always assumed it’s like vim now: only really maintained in theory, not in practice.
Please bear in mind that there are [now] two distinct rsync codebases.

The original is the GPL variant [today displaying "Upgrade required"]:

https://rsync.samba.org/

The second is the BSD clone:

https://www.openrsync.org/

The BSD version would be used on platforms that are intolerant of later versions of the GPL (Apple, Android, etc.).

So you not used vim or neovim in the last 10 years ?
To be fair, there was a roughly 6 year period when vim saw one very minor release. That slow development period was the impetus for the fork of Neovim.
I know. I use Neovim. But since that, and thanks to Neovim, Vim has speedup and got some improvements.
Time for neorsync.

That said, VIM 8 was terrific.