Hacker News new | ask | show | jobs
by ghshephard 4920 days ago
Add up your downtime over a three year period by relying on Git (or gmail, or AWS) versus the cost of trying to engineer some local-backup system, and the downtime associated with that going awry.

Outages happens - as long as we're talking hours a year, pretty much everyone but life-safety systems, critical infrastructure, and payment/high-traffic commerce sites are probably better off just letting third-party cloud vendors manage their systems. Take the downtime and relax.

(Now, if downtime consistently gets into the 10s of hours/year, it's time to look for a new cloud provider. )

6 comments

You make a very good point, but it took me about three minutes to build a git mirror which we can push/pull to, can re-configure CI to if we need to, and can be used to run a full deploy from on the company VPS server.

* Create an unprivileged account & set a password that you don't need to remember -> sudo adduser git

* Add your public key from your laptop to the unprivileged user's authorized_keys file -> sudo su git; cd ~; mkdir .ssh; vim authorized_keys - then copy and paste your id_rsa.pub to that file

* Repeat that for all public keys on your engineering team

* In git's home directory, git init --bare <name of repo>.git

* On your local machine, git remote add doomsday git@<DOOMSDAY SERVER HOSTNAME>:<NAME OF REPO FOLDER>.git

* git push doomsday --all

* On colleague's box, git clone git@<DOOMSDAY SERVER HOSTNAME>:<NAME OF REPO>.git

Let me know if there is a better way of doing this, or if it's monumentally screwed somehow.

Yup. Github going down barely breaks my stride, but for a real production outage (e.g. Heroku going down), I pour myself a tall glass of scotch and thank my lucky stars I'm not the one who has to scramble around tailing logs and restarting servers. I'm pretty sure their ops team is better at this than I am anyway.
It's not about downtimes and outages. It's incomprehensible to me how lax businesses are with their backups, especially business where their clients data is everything. Yes, the brave new world of the cloud seems tantalizing, but even there, data can and will be lost. Don't just use only one way / provider / service / mechanism for backing up your data.

A tape / lto backup system doesn't cost the world. Yes, it introduces overhead and maintenance, but I'd rather be safe than sorry.

At my place of work we currently use a lot of virtual servers, hosted by external providers. We use their backup and snapshot mechanisms. But we also pull all data to our local back up server. From there we backup to tape on a daily basis.

I do have backups of all my (relevant) GH repos since that's just a "git pull" away and can be automated nicely. But I'd probably be out of luck running my regular CI-process with github down or do a deployment. Both rely on having a public-facing git server - having a backup does not imply that I have a full git server running. I could set one up and administer it, but it's just too much effort given GH uptime.
Github is definitely in 2-digits this year.
I doubt it. (Assuming by 2 digits you mean < 99.0%, ie that they don't have "two nines" (though I guess two nines could even be 98.5 with rounding)).

1% downtime is over three days. They've had some big outages, but I think this may be their longest of the year and it was a < 6 hours. They could have one of those every month and still only have 72 hours of downtime, which is 99.18%.

I'm pretty certain he means 2 digits of hours, given the reference in the parent comment to 10s of hours of downtime
I agree with you but one thing: you don't have to build anything locally. Just pushing to bitbucket as backup would have done it. It does not have to be a locally hosted solution.
Periodic reminder that git != GitHub