Hacker News new | ask | show | jobs
by toomuchtodo 3929 days ago
Something to note: Unless you're storing data in us-east-1, all other regions in AWS are "one datacenter". Yes, they have AZs, but those aren't datacenters, they're just compartmentalized segments of the same datacenter.

So! If you can tolerate the loss of a datacenter, store in Blackblaze. If you need geo-redundancy until Backblaze can offer it? Store in us-east-1 (which is geo-redundant between Virginia and Oregon).

5 comments

This is incorrect on almost all points.

All AWS AZs are physically separated facilities with redundancy on all their infrastructure, although they're obviously in the same general area.

us-east-1 is not geo-redundant. It is entirely on the east-coast, as the name suggests. Although S3 does have geo-redundancy in all regions.

You may have been thinking of "US Standard", but it is the same as "us-east-1".

http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_r...

> This is a feature for EU and US-West. US Standard is bi-coastal and doesn’t have read-after-write consistency.

Quote from Jeff Barr @ AWS: http://shlomoswidler.com/2009/12/read-after-write-consistenc....

Their documentation is really inconsistent on this...
I've emailed Jeff to get clarification on it.
You did? I didn't see it...
From my gmail account :/ I'll resend!
Do you have a source that all other regions are "one datacenter"?

All I could find on the S3 FAQ says "your objects are redundantly stored on multiple devices across multiple facilities." which seems to contradict the "one datacenter" claim.

Also, do you have a source that us-east-1 is geo-redundant between Virginia and Oregon? That was not my understanding of how it worked.

AWS considers multiple facilities to be separate AZs in the same region. If you want multi region durability (besides us-east-1), you need cross region replication enabled (from the same FAQ you read).

"You specify a region when you create your Amazon S3 bucket. Within that region, your objects are redundantly stored on multiple devices across multiple facilities. Please refer to Regional Products and Services for details of Amazon S3 service availability by region"

Note, "within that region". Separate AZs, same geographic location.

"CRR is an Amazon S3 feature that automatically replicates data across AWS regions. With CRR, every object uploaded to an S3 bucket is automatically replicated to a destination bucket in a different AWS region that you choose. You can use CRR to provide lower-latency data access in different geographic regions. CRR can also help if you have a compliance requirement to store copies of data hundreds of miles apart."

This post http://shlomoswidler.com/2009/12/read-after-write-consistenc... has a quote from Jeff Barr at AWS indicating that us-east-1 is bicoastal, which is also why its eventually consistent, instead of immediately after a write (EDIT: it appears this constraint no longer applies to the US standard region).

I'm familiar with CRR.

I asked for sources about your "one datacenter" claim. Just because several facilities are in the same geographic region does not mean they are the same datacenter.

Just because something is bicoastal does not mean your data is replicated on both coasts. It could also mean that your data is stored on either the west or the east coast.

I would have trouble believing they store twice the data as their other regions but charge the same (actually a bit less!).

Multiple facilities != "one datacenter"
Like I asked below, do you have a citation?
"No one data center serves two availability zones" :

http://www.theregister.co.uk/2015/04/16/aws_data_centre_arch...

From your link:

"To solve latency, Amazon built Availability Zones on groups of tightly coupled data centres. Each data centre in a Zone is less than 25 microseconds away from its sibling and packs 102Tbps of networking."

25 microseconds at the speed of light (best case, through a vacuum; through fiber is significantly slower) is ~4.7 miles, and based on the quote, that is the furthest they are apart. If your buildings are within 1-2 miles of each other, they're essentially the same facility.

That is not geographically redundant.

> If you need geo-redundancy until Backblaze can offer it? Store in us-east-1

Or store it in Amazon AND store another copy in Backblaze. This isn't necessarily an "either/or" question. Having two copies with two different vendors in two separate regions is probably more reliable than having two copies inside the same vendor. For example, if Amazon has a large outage that affects both your regions, you can still access the copy in Backblaze.

If you're going to pick two providers, use Backblaze and Google. Google's Nearline Storage is still more reliable (AWS only offers a 98% SLA on a monthly basis for S3 IA storage class) and cheaper (if I recall properly) than AWS' Intermediate Availability offering.
This is incorrect. If you look for news articles about Amazon constructing data centers or buying facilities you'll notice that they have multiple data center facilities in each region.
Have a citation? My data is directly from AWS docs and speaking with AWS staff.
The docs you quoted specifically call out "multiple facilities".
what about Ireland? Amazon have 7 active Datacenters, 1 being build and 1 in planning...