[derefr]

> If your programmer makes a mistake and a line of code deletes some mission critical data from Amazon S3, then all the Reed-Solomon encoding in the world doesn't help you, the data is still gone.

I'm surprised at the implication here, that you'd use Glacier on a non-versioned bucket. Making destructive updates impossible doesn't cost much extra in archive fees.

[brianwski]

Ok, so let's say you forget to pay your Glacier bill because the IT guy left and the credit card changed and the alert emails go nowhere. Bye-Bye-Glacier! No payment, no customer data, Amazon might delete your data due to a tiny administrative screwup.

My point stands: if you don't mind losing your data, store it in one vendor. But if you would REALLY lose your business and put 10 people out of work if the data is lost, storing it in Amazon (or Backblaze) without a second copy backed up somewhere else and a third copy backed up in yet a third location (with a totally different vendor with a totally different payment system) is irresponsible.

[derefr]

But... let's say you have three accounts with three storage companies. Now let's say you outsource the management of those accounts through one company... or even, equivalently, delegate it to a subsidiary or partner of your company. And then you accidentally stop paying them, or they can't requisition the budget necessary to pay the providers, or whatever. Now you're still stuck, even though you're nominally doing things "in-house."

What you actually need is a provider that will guarantee the durability of your data even if they (temporarily) cut off your access to it for lack of payment[1]. Anything else is just a level of indirection that suffers the same problems.

---

[1] I don't actually know if anyone does this, let alone AWS. Here's a quote from Tarsnap's FAQ—where you'd think cperciva is someone who would have considered the "I had no idea my infrastructure was relying on this service until it shut off" use-case:

> You will be sent an email when your account balance falls below 7 days worth of storage costs warning you that you should probably add more money to your account soon. If your account balance falls below zero, you will lose access to Tarsnap, an email will be sent to inform you of this, and a 7 day countdown will start; if your account balance is still below zero after 7 days, it may be deleted (along with any data you have stored) at our discretion. (If you can't add money yet but will be able to later, contact us and explain the situation. We're reasonable people and simply knowing that you're alive and haven't forgotten that you were using Tarsnap is very helpful.)

7 days is probably reasonable in the case where there's an active IT staff who will notice when, say, servers stop backing up. But if nobody's watching for that...

[cperciva]

7 days is probably reasonable in the case where there's an active IT staff who will notice when, say, servers stop backing up. But if nobody's watching for that...

Do you have a solution in mind for the case of a company where email is going to /dev/null and nobody is reading the output of their cron jobs?

I mean, if I can't contact someone, it doesn't really matter if I wait a week or a month...

[derefr]

I'm not sure; I think the main thrust of the solution would be getting the users who really, really care about their backed-up data (which should be most of the people who are using a back-up solution) to do some extra stuff up-front to insure it:

• Ask people to provide optional contact information for an "executor of their estate"—a person who can make decisions about what happens to their data on their behalf if they cannot be reached.

• Ask people for a secondary credit card that can be charged as a backup: specifically, suggest that this be the personal card of Someone Important in the company, who will be likely to notice the charge and flip out.

• Ask for a flat-fee deposit to enable a secondary "long-term storage, no uploads, no monthly billing" mode of usage. Make it enough money to be motivating when you imagine just schlepping this hunk of data around for the rest of your life. If the user has paid this deposit, and their regular card gets declined, switch them to this mode and consume the deposit. If they close their account, refund the deposit if it hasn't been consumed.

And so forth.