Hacker News new | ask | show | jobs
by rhencke 1951 days ago
I grew up in an era where running your own servers was just generally accepted practice.

It's been fascinating watching how dramatically that viewpoint has shifted over the years to the point where it is now a novel idea to do so.

5 comments

I worked at a startup that managed its own bare metal. It was very hard and time consuming. And when things went down it was on you. You needed a really good datacenter partner to make sure that they were on top of things and could drop things at a moment's notice. But you don't know how responsive they will be until you're actually experiencing an outage.

The level of convenience that cloud providers give is just orders of magnitude more efficient and easier.

I have worked for companies in the past with horrible datacenter partners (one of them did not know that one of the two switches they routed our traffic through was completely dead), so I definitely agree with you on that.

On the flip side, I wonder how much better the support from a cloud provider is if it's an isolated problem and not something that's setting twitter aflame.

If it takes a cloud provider in the order of hours to get me back online, I could probably get the same sort of service from one of the better colos/hosting providers, especially if they were local and I had the ability to make a call to get support.

There are other conveniences to cloud providers of course, but I think I if I could find highly skilled ops people and pay them well, I would run my own servers every time. For the kind of games I've worked on, the money/CPU cost of cloud is ludicrous.

The trick these days is even finding high-level ops people who aren't already working 3-400k jobs for AWS/Azure/GCP

> On the flip side, I wonder how much better the support from a cloud provider is if it's an isolated problem and not something that's setting twitter aflame.

I’ve had 4w to resolution for major service degradation on a big name provider and ~2d for full site outage (highest support tier)

You said it yourself - aws, goog, msft have very good sres but they’re not your sres. Meaning they dont care if you have a big event/demo/deal close coming up before they start doing network gear upgrade and such...

You make me feel so underpaid, is 300k the going rate for being a DevOps Engineer that knows the old ways of CGI/LAMP and can leverage running Exchange/IIS nowadays?

I don't even know if these salaries are truthful now.

Usually the high DevOps salaries like that are for roles that involve high scale, or trying to fuse old tech with new infrastructure.

Those are certainly in range for DevOps people at the higher end of the scale.

I don't actually know if that's the going rate TBH. That was the range our ops (not devops, he managed our datacenter ops) guy was hired away for almost a decade ago
An enterprise client of a cloud vendor will have dedicated account technicians - response time is extremely fast (minutes or hours). Perhaps things have improved compared to some of the other comments here.

The other consideration is that things generally don't just 'go down' like they do with bare metal (because HA - replication and so on), but if they do, it's likely affecting a large portion of the internet too.

It surprised me how well cloud providers have managed to sell "availability" and lack of responsibly for site downtime, and how much of a premium they can charge for it. Most people don't need 5 9's (not that any cloud actually provides that level of reliability) or infinite scalability, just a box that's good enough in both dimensions.
I switched my personal website to a Raspberry Pi 4 in my basement a while ago, with cloudflare caching and roll-your-own DDNS - I’ve been pleasantly surprised by how easy the whole thing was to do (although my requirements are far from exotic).
Any good resources to share?
Not the poster but I assume an easy setup is simply ubuntu server, nginx, get a domain and put it on cloudflare free plan, setup a script that updates your home IP to cloudflare dns records. Then do letsencrypt for the https cert, setup automatic renewal, and you're probably good.
Bang on - that’s exactly my setup. Only thing I’m also using is PM2 for node script management.
Any good resources on how to go about implementing this?
I wonder how much of this is just the industry maturing/specialising. We don't think it's weird that most people don't mill their own flour when they bake bread, so long as the quality of the flour is good enough we're happy for someone else to do it for us.

In the same way most people/companies don't really need to care what hardware their application runs on, only that it meets some bar of quality/cost that's appropriate for them. If someone else is delivering this then you've removed a small department's worth of overhead/planning from your corporate structure.

If you need to have your baked goods constaltly, you will be better prepared with many flour providers in case of that one provider does NOT LIKE YOUR RECIPE and stop selling the ingredients to you... Or get your own flour mill.

So, not a ideal analogy...

Pissing off your supplier is something everyone has to be careful of. If you're drawing little swastikas on your cakes while telling everyone how great "Phil's flour" is then don't be surprised when Phil doesn't want to associate himself with you any longer.

The sibling comment pointed out that vendor lock in can be a problem which I agree with, but I think for most of the industry that's a problem of protecting yourself from predatory price hikes/services being deprecated rather than the problem of actively pissing off people you need.

The difference is that flour has been completely commoditized, while cloud hosting has only recently started becoming commoditized and isn't all the way there yet.
I think your underlying point is fair, but I'd like to see you post a more constructive explanation of where it would be useful for the analogy to capture a truth about cloud services that is not true for, say, flour - switching cloud providers is incredibly difficult due to vendor lock-in.
Yeah... and the crazy thing is that people on the big clouds or even just using Firebase have experienced outages as a result.

I've had zero downtime due to running on a Digital Ocean VPS the past few years and very, very brief outages due to my own decisions around various upgrade and backup decisions. I spend less, I get full control and there's zero risk of a surprise bill.

I think the key thing is that most customers won't blame you if your service goes down due to an AWS outage.