[kalleth]

In most startups? You're mostly correct.

But you still have some risks here, yes, with a super low probability, but a company-killing impact.

In some industries - banking, finance, anything regulated, or really (I'd argue) anywhere where losing all of your data is company killing - you will need a disaster recovery strategy in place.

The risks requiring non-AWS backups are things like:

- A failed payment goes unnoticed and AWS locks us out of your AWS account, which also goes unnoticed and the account and data are deleted

- A bad actor gains access to the root account through faxing Amazon a fake notarized letter, finding a leaked AWS key, social engineering one of your DevOps team, and encrypts all of your data while removing your AWS-based backups

- An internal bad actor deletes all of your AWS data because they know they're about to be fired

...and so on.

There's so many scenarios that aren't technical which can result in a single vendor dependency for your entire business being unwise.

A storage array in a separate DC somewhere where your platform can send (and only send! not access or modify) backups of your business critical data ticks off those super low probability but company-killing impact risks.

This is why risk matrices have separate probability and impact sections. Miniscule probability but "the company directors go to jail" impact? Better believe I'm spending some time on that.

[cameronh90]

Just to add that S3 supports a compliance object lock that can't even be overridden by the root user. Also AWS doesn't delete your account or data until 90 days after the account is closed.

Between these two protections, it's pretty hard to lose data from S3 if you really want to keep it. I would guess they are better protections than you could achieve in your own self managed DC.

I'm guessing AWS has some clause in their contract that means they can refuse to deal with you or even return any of your data if they feel like it. Not sure if that's ever happened, but still worth considering it.

[manquer]

Yes threat models is obvious qualifier, if you have a business that requires backup on the moon if there asteroid collision then by all means got for it.[1]

For most companies what AWS.or Azure offers is more than adequate.

An internal bad actor with that level of privileged access can delete your local backups or external one can all things you he can do to AWS he can likely do easier to your company storage DC too.

Bottom-line it doesn't matter if customers can pay for all this low probability stuff that can only happen on the cloud and not on Prem sure go ahead. Half the things customers pay for they don't need or use anyway.

[1] assuming your business model allows you to spend the expense outlay you need for the threat model

[chrisandchris]

Nope. 3-2-1 strategy. 3 Backups, 2 Medias, 1 Offsite. Now try to delete files from the media in my safe. Only I have a key.

Sure, your threat model may vary. But relying on cloud only for your backup is simply not enough. If you split access for your AWS backup and your DC backup to two different people, you mitigated your thread model. If you only have 1 backup location, that's going to be very hard.

[manquer]

All of these are questions asked and solved 10 years ago by bean counters who only job is risk mitigation.

Every cloud provider has compliance locks which even root user cannot disable, version history and you can setup your own copy workflow storage container to second container without delete/update access to second one to two different people or whatever.

You don't need to do any of it offsite.

[JackFr]

Not sure I agree about the usefulness of different media.

Having had to restore databases from tapes and removable drives for a compliance/legal incident, we had a failure rate of >50% on the tapes and about 33% for the removable drives.

I came away not trusting any backup that wasn’t on line.

[lambic]

We have AWS backups, "offsite" backups on another cloud provider, and air-gapped backups in a disconnected hard drive in a safe.

The extra expense outlay for the 2 additional backups is approximately $50/month, so it's not going to break the bank.

[manquer]

Egress from aws is not cheap.

At $50/month scale a lot of things are possible. Most companies cannot store their data in a hard disk in a safe. If you can, then cloud is a convenience not a necessity for you. I.e. you are perfectly fine running your storage stack for the most part.

My company is not very big(100ish employees) and we pay $200k+ for AWS in just storage and AWS is not even out primary cloud. If we have to do what you have, it is probably in bandwidth costs alone another $500k. Add running costs in another cloud and recurring bandwidth for transfers , retrieval from Glacier for older data on top of that.[1]

Over 3 years that would be easily $1-$1.5 million in net new expenses for us scale.

No sane business is going to sign off on +3x storage costs on a risk that cannot be easily modeled[2] and costs that cannot be priced into the product, just so one sysadmin can sleep better at night.

[1]your hard disk in a safe third component is not sensible discussion point at reasonable scale.

[2] this would be probability of data loss with AWS * business cost of losing that data > cost of secondary system.

Or probability of data availablity event(like now) * business cost of that > cost of an active secondary system .

For almost no business in the world the either equation would be valid.

For example even the cost is 100B dollars in revenue with 6 nines of durability the expected loss would be only $10,000 (100B * 0.000001) a secondary system is much costlier than that.

[lambic]

> My company is not very big(100ish employees)

I don't get how this is relevant at all, it's more about how much data your company stores than how many employees it has.

I've worked for a company with 5000 employees that stored less data (fewer data?) than my current employer that has less than 100.

> No sane business is going to sign off on +3x storage costs on a risk that cannot be easily modeled

Probably not, but for us the cost is about 0.1x our aws storage costs, so it's a no-brainer.