Hacker News new | ask | show | jobs
by iknownothow 709 days ago
How does it compare with s5cmd [1]? s5cmd is my goto tool for fast s3 sync and they have the following at the top of their Github page:

> For uploads, s5cmd is 32x faster than s3cmd and 12x faster than aws-cli. For downloads, s5cmd can saturate a 40Gbps link (~4.3 GB/s), whereas s3cmd and aws-cli can only reach 85 MB/s and 375 MB/s respectively.

[1] https://github.com/peak/s5cmd
1 comments
Starofall 709 days ago
I have not yet compared it against this tool, but the given numbers are for download and not for syncing files.

For a S3->S3 sync using an C6gn.8xlarge instance, I got up to 800MB/sec using 64 workers, but the files were only in average around 50MB. And the bigger the file the higher the MB/sec.

Also from my short look into it, s5cmd does not support syncing between S3 providers (S3->CloudFlare).

link