[ltbarcly3]

Buying rack space at a colo costs money, but if you are spending millions of dollars on AWS you will likely end up spending a few hundred thousand including a salaried sysadmin to manage the hardware.

This does mean increased management complexity, so you have to build out an operations team. The total for salaries will be around 400-600k.

In the end you will have some setup costs and you will have to choose a subset of the features AWS offers, but you'll save millions of dollars per year and have much better performing hardware and much, much more flexibility.

AWS is extremely expensive.

[boulos]

Disclosure: I work on Google Cloud.

The blog post doesn’t make it as direct, but one of their biggest costs was for networking between datacenters (Availability Zones in AWS). Most comparisons for “buy a rack at a colo” assume one colo, and a static fleet of hardware.

If you wanted to compare apples-to-apples, you’d need to have (at least) three nearby colos with enough capacity to handle one going down entirely at peak load (“N+1”). Leased lines in a metro area aren’t actually all that expensive, but like the compute, you also need to purchase that with failure in mind.

tl;dr: Maybe, but the analysis needs to assume the same(ish) reliability outcome. Otherwise, they could have avoided lots of cost by just running in a single Zone.

[pm7]

> If you wanted to compare apples-to-apples, you’d need to have (at least) three nearby colos with enough capacity to handle one going down entirely at peak load (“N+1”).

Not true if it's possible to fallback to cloud. That way we can have both high reliability and low cost (other then during outage/maintenance of collocation).

[boulos]

Hmm. I read the comment as saying “no cloud, because you’ll save so much by just being on-prem”. And I think an “apples-to-apples” comparison requires an N+1 setup including both compute and networking.

Hybrid could be many different setups, but before their “zonal affinity” change it would actually be worse, right? (Egress over Direct Connect is 4x higher than Zone to Zone, while “internet” egress is 8x). What are you assuming for the balance of Compute and Networking across at least three “sites”?

[pm7]

> Hmm. I read the comment as saying “no cloud, because you’ll save so much by just being on-prem”. And I think an “apples-to-apples” comparison requires an N+1 setup including both compute and networking.

That is valid interpretation. I just wanted to say that is you need high availability it might be cheaper to have one colocation and cloud in standby.

> Hybrid could be many different setups, but before their “zonal affinity” change it would actually be worse, right? (Egress over Direct Connect is 4x higher than Zone to Zone, while “internet” egress is 8x).

Yes, in/out traffic would be one of more problematic points of such setup, but there should be some solutions available (BGP?).

> What are you assuming for the balance of Compute and Networking across at least three “sites”?

Least expensive should be zero compute in cloud unless there is issue with collocation. Depending on specific scenario, some storage/databases would have replication to cloud. I don't know how I would setup networking in such case.

[pm7]

> Least expensive should be zero compute in cloud unless there is issue with collocation.

One more thing: cloud can be great to scale up in peak utility without buying servers that will idle most of the time. It's just that using only cloud might be much more costly, even if it is easier.