Hacker News new | ask | show | jobs
by spullara 5534 days ago
Also worth noting is that Amazon isn't forcing you to use EBS. They also have tons of fast local storage available to RAID as you wish.
4 comments

I strongly believe one of the most positive aspects of EC2 was that it demonstrated a beautiful philosophy that a node and their disks should not be relied upon to always be around and pushed it into the mainstream.

Even for people who didn't use EC2 the existence of the platform caused more people to rethink their architectures to try to rely less on Important Nodes.

EBS is a step back from that philosophy and it's a point worth noting.

One of the great things this post does is enumerates some of the underlying reasons why relying on EBS will inevitably lead to more failures and in ways that are harder and harder to diagnose.

> EBS is a step back from that philosophy and it's a point worth noting.

Amazon doesn't use EBS itself, right? Isn't EBS something that AWS allowed its customers to nag it into against (what it considers) its better judgement?

Yep. And this may be one of those cases where they would have been better off ignoring their customer requests for the good of their reputation and their customers uptime.
I agree. We run all of our MySQL and Mongo slave servers with local RAID-0 ephemeral storage. One dies? So what, we remove it from the pool and provision another.

Only our master and our slave backup server runs on EBS. We aren't as write oriented so we can live with some of the limitations of EBS, but we've even considered moving our master MySQL and Mongo servers to ephemeral storage and just relying on our slave back database server to run on EBS (for which we take freeze/snapshots of often). That server rarely ever falls more behind in relay updates.

Do you run xlarge to get the extra ephemeral disks?
Do you mean RAIDing local instance storage? I haven't heard of this approach so far, do you have any links?
Depending on the instance that you have provisioned, you have anywhere from 1 to 4 ephemeral (local) disks available to your instance.

They are typically available as /dev/sd[bcde]

In centOS, implementing a RAID-0 block device across 2 ephemeral disks that is present on an m1.large instance can be done via the following:

mdadm --create /dev/md0 --metadata=1.1 --level=0 --quiet --run -c 256 -n 2 /dev/sdb /dev/sdc

You'll then need to format the block device with your fs of choice. Then mount it from there.

Just remember the super important keyword:

ephemeral

WHEN your EC2 node disappears (and it will), you will lose everything on that RAID.

That's not a bad thing if you know it'll happen and plan for it, but do be aware of it.

Except Amazon does... Their free micro instance only supports EBS. For some of us who just want to dip our toes into AWS without too much money or commitment, EBS is our only choice.
Then you really don't need to worry about fault tolerance or uptime.