Hacker News new | ask | show | jobs
by lowbloodsugar 3821 days ago
If you are pulling large files from S3 we have found that they can be sped up by requesting multiple ranges simultaneously. It is easy to hit 5Gb/s or 10Gb/s on instances with the necessary bandwidth, accessing a single file, or multiple files. We have not encountered a limit on S3 itself. YMMV.
1 comments

Excellent https://github.com/rlmcpherson/s3gof3r is my tool of choice for "fast, parallelized, pipelined streaming access to Amazon S3."

If you want to saturate network bandwidth with S3 that's the one tool I know that can do it.