Hacker News new | ask | show | jobs
by threatripper 1893 days ago
Do they share infrastructure?
2 comments

Given the fact that it's also down, I'd say yes
It could be some rouge admin who actually deleted Facebook.
At that size of a company there's usually blast radius restrictions and per-role permissions. I don't expect anyone has enough rights to "delete Facebook" on their own.
I guarantee you that there are 100+ people who could take Facebook down for 24+ hours if they went rogue.

For example the people responsible for the bootup scripts of Facebook infra could sneak in a "0 0 1 * * /bin/rm -rf ${TEMPDIR}/*" into crontab... They'd set the commit message as "clear out temp monthly" and it would get deployed across the entire fleet till in the first of next month every disk at Facebook gets erased because TEMPDIR isn't defined...

I guess they have enough pending stock to deter them...

This wouldn't "delete" Facebook or many much smaller companies. It would result in maybe a small outage and get restored immediately in most cases. It's also an infra change you'd need across many systems - this isn't possible as a single change "across entire fleet".

This is not how non-trivial services work.

That’s...now how any of this works. You can’t just change integrity-bearing things without FIM systems kicking in. And you’d need collusion to get something mainlined that would bypass that.
You don't need collusion - just a code reviewer not paying proper attention.
How though? Every past author of that script would be notified of such a change. It'd be insane if all of them would pretend they didn't see it and accept that change.
It would be hidden amongst a big refactor, and it would have a bunch of unit and integration tests that all work correctly (because they do set the environment variables correctly).

Writing code that appears to do one thing and actually does another is very doable - you only need to hide one malicious line amongst thousands in a code review, while the reviewer needs to inspect every line.

Besides, the vast majority of code reviews are 10 minutes or less.

Facebook servers do not have cron installed.
Example using widely understood tooling... But there are plenty of other one line changes with similar impact
one could hope
Like a moulin rouge admin?
You got me there :D
Time for them to hit the gym and call a lawyer!
Yeah, that's what I was wondering. Strange that there is a SPOF for Facebook/Insta/Whatsapp.
I had the impression that they are still run like mostly independent companies who only share some data on the backend.
AFAIK, I remember reading something about Instagram moving into Facebook datacentres some time ago. I believe they were on AWS before the acquisition.
Yeah that's what I was thinking too, so this is probably DNS related.
Might I ask what’s SPOF?
Single Point of Failure.
Thanks
Single Point Of Failure
Ta