Hacker News new | ask | show | jobs
by justapassenger 1304 days ago
Outages of complex systems tend to be complex. It’s very likely that institutional knowledge of how to resolve them is gone.
2 comments

Care to wager?
If it’s a conceptually complex system then the devs weren’t really that good in the first place. Yeah, technical debt happens, but you can’t have both that the system had good technical leadership and it can’t possibly stay up under replacement employees. (My impression is that it will stay up.)
What magical unicorn of a place have you worked at that could survive multiple entire teams quitting at once? Do you think the induction manuals literally jump of the shelf and explain themselves to the new hires that there isn't an HR team to hire, a security team to grant their access and a department to welcome them?

This is like the flight crew being raptured. The plane is going to crash.

This is the ground crew being raptured, the flight crew all exploding, the plane had an engine removed "to trim the fat", the company who sold you fuel pulled out of the deal, everyone on the plane who has ever played flight simulator dies and is replaced one of those people who know just enough to fuck up.
All could, and one of them passed through full employee replacement when it failed to raise a round of funding. They would have to suffer and hire good people, not miscellaneous cogs, but they would survive.
> If it’s a conceptually complex system then the devs weren’t really that good in the first place.

Spoken like someone who has never worked on a sufficiently complex system.

There's a possibility that it's a Rube Goldberg machine written by bad devs that is actually crap and deserves a rewrite, but it's the thing that currently Elon has. If it goes down and there's no one left to bring it back up, how is that not a disaster for him? Is he going to say "Just give us 3 months folks, we'll be back online with much better code, I've got the hardcore devs right h..."

Actually he'll promise a self-coding website within 1 month.. Another month, promise.. Just pay us 8 bucks now and you'll get it very soon!

Hum, idk, it depends on who left. I think I agree that never coming back up is a tall order. There ought to be enough competent people left, but I’d bet something like Twitter will set a record (at least for the past decade) for downtime in the next year