Hacker News new | ask | show | jobs
by bstpierre 5125 days ago
Most companies I've worked for have had some kind of annual fire drill / alarm testing. They announce it the prior week, and then, say, Tuesday at 10am the alarm goes off, everyone files out of the building into the parking lot for 5 minutes, then back inside. In 15+ years (at several different companies), only once has there been an actual fire department call where the evacuation was "real" (even then, there was no actual fire).

In those same 15+ years, mostly working for startups, there have been numerous drive failures. Unfortunately, failure (a) to verify backups before there's a failure, and (b) to practice restoring from backups has often meant that a drive failure means loss of several days' worth of work. In one instance, the VCS admin corrupted the entire repo, there were no backups, that admin was shown the door, and we had to restart from "commit 0" with code pieced together from engineers' individual workstations. That was when I got religious about making & testing backups for my work and the systems I was responsible for...