Hacker News new | ask | show | jobs
by boole1854 1619 days ago
Without warning and without contacting us first, Digital Ocean blocked public IP access to one of our critical production servers on a Friday evening on the basis of a third-party security firm reporting to them that we were hosting a phishing page. We were, in fact, not hosting a phishing page. If DO had done minimal due diligence before blocking us they could have confirmed this. However, it took until the next Monday before DO support responded to our explanations that the phishing report was a false positive and unblocked our IP.

We moved to AWS.
3 comments
start123 1619 days ago
Yes, I second this. It was a nightmare for me. They brought down the droplet and the recourse was that I had to create a support ticket. It was weekend and they took more than 24 hours to answer.

Until then, my site was down with customers complaining to me.

link
boole1854 1619 days ago
We ended up launching another droplet, setting up a reverse proxy from the new droplet to the blocked droplet (which was still accessible via internal IP), and updating DNS to point to the new droplet's public IP. At least that way we were not down all weekend. Nevertheless, we decided that DO support's kill-first-ask-questions-later policy was completely unacceptable.
link
riffic 1619 days ago
You didn't have a disaster recovery plan in place to spin up new critical production environments in the event of an outage impacting external network accessibility? This is on you as an Ops team, not really something to blame DigitalOcean for.

AWS will expect you to do the same thing but it'll be much harder just because it's AWS.

link
boole1854 1619 days ago
We did have a disaster recovery plan in place, and we implemented it.

It is my opinion that killing public access to a paying customer's server without talking to them first, and not providing any remedy for 60+ hours, is definitely something that DO support deserves blame for.

link
riffic 1619 days ago
from the other reply it sounds like you made up your disaster recovery scenario on the fly.

DigitalOcean (and a lot of IaaS providers like them) expect you to create new servers on demand because the underlying hosts aren't 100% reliable. It could have been a RAID failure or a bad PDU feeding a rack or a number of other scenarios that would have taken your droplet offline, not just an abuse ticket.

link
waffle_maniac 1619 days ago
Do you have any recourse to sue the third-party security firm? Or Digital Ocean?
link
riffic 1619 days ago
there's a limitation of liability written into DigitalOcean's terms of service.
link