Hacker News new | ask | show | jobs
by edude03 15 days ago
As always it depends but at least for me in my anecdotal experience as a Distributed systems engineer (I know, I'm clearly very biased here) - the problem with a single system is that mundane things cause unavailability like - software updates, hardware upgrades/failures, having a single fault domain (IE, someone writes a bad query that wedges the DB, now no one can use it) and since failure isn't built into the design, _when_ things fail the mean time to recovery tends to be rather high. For example, if the power supply explodes and takes out your server, how long would it take to procure a new server and restore from backup? On the flip side, if your system is designed around (for example) server less functions or spot instances- which become unavailable multiple times per day, you would have already engineered in fast recovery
1 comments

Well if the company have somebody like you it helps tons!

In the flip side:

> the problem with a single system is that mundane things cause unavailability

Is multiplied by each "system" you add. The basic failures are relative easy to deal and understand, the ones introduced by more complex system who knows?

Probably the most important step that I miss:

> How long would it take to procure a new server and restore from backup?

.. and applied to any complex extra you have.

For single system is viable (with something like nix) to go off for maybe half hour for what I see around, most of the time in procuring another machine (that what people do is to Bring any other machine it can not go to amazon and buy!) and restoring the backup.

Of course, I factoring that downtime is not "seconds or minutes" here, but neither I think many can do like that

P.D: all

I didn’t make the point I was trying to if

> Is multiplied by each "system" you add

Was still the take away. The idea is the lost a single instance should cause downtown so it’s not multiplied. IE three instances of the database case tolerate a loss of a single instance for maintenance since the other two will take over the load.

There is of course an argument against the cost of this and sure I’d even accept the complexity argument since you probably need to add more tooling to manage the hand over but again the point of this complexity is specifically to avoid having a single point of failure / to naturally handle failure such that the possibility of downtime isn’t multiplied