Hacker News new | ask | show | jobs
by epi0Bauqu 5533 days ago
How do you move away from EBS and still deal with large data?
2 comments

Not sure what you had in mind by "large", but instance storage goes up to 1.7TB: http://aws.amazon.com/ec2/instance-types/
The reason Reddit uses RAID10 is for performance, not disk size. A single instance storage device is just too slow for the Reddit database.
Many instance types have 2 or 4 virtual disks (presumably on different physical disks).
I imagine they'd do consider some combination of the following (sorted by most likely)

1. Sharding data 2. Pulling tables out to other servers from the main DB 3. Pruning excessive data 4. Compressing data

It still has to be stored somewhere though right? If it's EBS you've just made yourself a complicated solution that will eventually fail all over again. No?
If the data is sharded, then the data/server is small enough enough to fit within the individual server's disk and you no longer need EBS to store it.