Hacker News new | ask | show | jobs
by toomuchtodo 3720 days ago
TL;DR

Newer, larger "snowball" appliance (current = 50TB and newer 80TB version), as well as access in additional international regions.

Network optimized S3 uploads (faster inbound transfers) for an extra 4 cents/GB.

Sidenote: I'm curious if you can get the same speedup for inbound transfers by using a Cloudfront distribution in front of your S3 bucket for no additional charge (no cost for inbound transfer on either service).

3 comments

Even though their Amazon S3 Transfer Acceleration Speed Comparison page [1] do test uploads only, this new feature is documented to accelerate downloads as well.

I guess optimized WAN network routing between S3 regions and their edge locations usually beats regular internet routing.

[1] http://s3-accelerate-speedtest.s3-accelerate.amazonaws.com/e...

I didn't mention that because it wasn't in the official blog post, but you're correct.

My assumption is that AWS has their own dedicated fiber/private connectivity between edges and regions.

It is mentioned, although in only 1 tiny instance:

> With today’s announcement, the edge network also helps to accelerate data transfers in to and out of Amazon S3.

We played with this a bit using both cloudfront and other cdns. While we did see some small improvements, it wasn't nearly as good as we were expecting on the upload side. It may be fixed now, but there were also many issues getting the proper headers to pass through cloudfront to s3 for chunked uploads.
This might not be the right question, but:

Did you test using S4CMD with many threads as opposed to S3CMD?

You can't upload through Cloudfront. It's a good analogy though, you can think of this feature as a "CDN for uploads", if you'll excuse the abuse of the CDN acronym.
You cant? PUT/POST through Cloudfront supported since October 2013.

https://aws.amazon.com/about-aws/whats-new/2013/10/15/amazon...

"You can use existing Amazon CloudFront distributions for upload requests by simply enabling support in the AWS management console. When end users upload content, CloudFront will send the upload request back to the origin web server (such as an Amazon S3 bucket, an Amazon EC2 instance, an Elastic Load Balancer, or your own origin server) over an optimized route that uses persistent connections, TCP/IP and network path optimizations."

I stand corrected, thanks for the info. Cloudfront doesn't appear to support multi-part uploads however, which I guess is the primary value-add of the newly announced service.
You're welcome. And you're correct. As long as you don't need to upload multi parts in parallel or objects over 5GB, standard upload works fine.
Without multipart upload, you can't resume uploads. Even with moderate sized files that's a bummer.
That gets less important each passing year as connections get faster, even on mobile. I agree you can't restart uploads though without multi part support.
Can you elaborate on why you came to this conclusion? I am able to do multipart uploads via cloudfront to s3.