[manigandham]

The entire value of these companies is in the outsourcing of the entire problem.

It's incredibly hard to have multiple vendors if you're using specific or advanced functionality, and if you're hosting a failover yourself then you might as well just use that instead.

[macspoofing]

>It's incredibly hard to have multiple vendors if you're using specific or advanced functionality, and if you're hosting a failover yourself then you might as well just use that instead.

I'm not suggesting any specific solution and I didn't even suggest they run their own infrastructure. There are good reasons to outsource these kinds of operations. But what happened was the company got the rug pulled underneath their feet and realized they didn't know how to contact their critical supplier. That's not good and that's their failure in this mess.

I don't understand the resistance to identifying and mitigating business risks. Honestly.

[manigandham]

You're not understanding because there is no resistance. This was the risk.

[macspoofing]

Again, let's say Twilio is the only supplier that could have provided this service for them, they still need to understand what risks that entails and how to mitigate them.

What happened in this situation is that Twilio for one reason or another decided to block access to their service, ostensibly halting their business. OK. That's not good. You know what made it worse? The CEO realizing that they don't have any line of communication to Twilio and that Twilio had shit tech support and therefore he had to frantically trawl Twitter and HN and Reddit and emailing Jeff Lawson (Twilio) CEO. Are you telling me they couldn't prevent this? You sure there isn't an Account Manager assigned to their account? Twilio holds conferences, there are plenty of opportunities to forge some relationship with someone at Twilio so you could at least backchannel issues like this (in fact, some Twilio dev here at HN noticed this and escalated it). By the way, Twilio is not the only telephony provider. If Twilio has shit support, OP can find another critical supplier, maybe a smaller or a more expensive vendor that is more responsive.

How about this: I looked through OPs comment history and he mentioned that their support numbers is provided by their service (i.e. they dogfood their product). Is that not a risk? If they are down, or Twilio is down, their customers can't reach a human either.

Very few of us control our circumstances, but we can certainly control our response to them. Outsourcing isn't the problem here. It has benefits and detriments. Lack of planning, and foresight is the issue here. This could have been a 10 minute outage, instead of a full day outage.