Hacker News new | ask | show | jobs
by kjellsbells 341 days ago
Perhaps reframe the problem not as data loss because S3's technical infrastructure failed but because of one of the many other ways that data can get zapped or that you might need it. For example:

- Employee goes rogue and nukes buckets.

- Code fault quietly deletes data, or doesnt store it like you thought.

- State entity demands access to data, and you'd rather give them a tape than your S3 keys.

I agree that with eleven-nines or whatever it is of availability, a write to S3 is not going to disappoint you, but most data losses are more about policy and personnel than infrastructure failures.

4 comments

This is solved for using versioning with MFA for delete or corruption risk, S3 export if required to provide a copy. Data can also be replicated to a write only bucket in another account, with only the ability to replicate.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/MultiF...

https://docs.aws.amazon.com/AmazonS3/latest/userguide/object...

Yep, for many applications, versioning is the lightweight solve.

But.. aws backup is still nice, if a bit heavy. I like common workflows to restore all stuff (ddbs, managed dbs, buckets etc) to a common point in time. Also, one of the under-appreciated causes of massive data loss is subtly incorrect lifecycle policies. Backup can save you here even when other techniques may not.

AWS Backup can get really pricey since you pay GB-month for every single restore point. Plakar only charges once for the initial backup and then for the small deltas on whatever cheap storage you pick.

Also, AWS Backup locks your snapshots into AWS vaults, whereas Plakar lets you push and pull backups to any backend—local disk, S3, another cloud, on-prem, etc.

AWS backup is a bit more nuanced than that; ideally the thing you want is N days of PITR (point in time recovery) and you want that across all your data stores (RDS dbs, buckets, dynamodb tables, etc etc), and you want to be able to restore them all to a common point in time. 7 or 30 days or PITR are common choices. It is ideal if you can perform a data restore in 1 operation since your hair may be on fire when you need to use it. In practice almost all your recovery will be from this.

The storage needed for this depends on the data change rate in your application, more or less it works like a WAL in a DB. What is annoying is that you can't really control it (for obvious reasons), and less forgivably, AWS backup is super opaque about how much is actually being used by what.

Retention of dailies / weeklies / monthlies is a different (usually compliance) concern (NOT operational, not really, if you have to restore from a monthly your business is probably already done for) and in an enterprise context you are generally prevented from using deltas for these due to enterprise policy or regulation (yeah I know it sounds crazy, reqs are getting really specific these days).

People on AWS don't generally care that they're locked in to AWS services (else.. they wouldn't be on AWS), and while cost is often a factor it is usually not the primary concern (else.. they would not be on AWS). What often IS a primary concern is knowing that their backup solution is covered under the enterprise tier AWS support they are already paying an absolute buttload for.

Also stuff like Vault lock "compliance mode" & "automated restore testing" are helpful in box-ticking scenarios.

Plakar looks awesome but I'm not sure AWS Backup customers are the right market to go for.

A fun one I've seen before: Your encrypted content reused a KMS key that was provisioned by a temporary CloudFormation stack and got torn down months ago.
Accidental crypto shredding? Oof.
S3 provides an object lock in compliance mode when nobody at the organization including its admins can delete objects during the specified period.
S3 buckets can just vanish for lots of reasons. With AWS’s shared-responsibility model, you’re the one who has to back up and protect your data not AWS.
The hold in compliance mode with AWS is accepted way to persist data that a company obliged to hold legally by US requirements.

And if your company has a sale contract with AWS the buckets cannot just vanish or AWS cannot close the account at arbitrary moment.

Or: AWS closes your account with a vague reason ("you violated our terms, we won't tell you which one") with no way to appeal.