[vidarh]

From having managed several racks worth of equipment in two separate data centres 1-2 hours travel from my office at the time: I cost closer to that $200k/year, but I also only usually visited the data centres 2-3 times a year, and other than that we used "remote hands" at the colo to do maintenance and be on-call 24/7. On ~60+ servers, we had maybe on average one minor incident every couple of months that required physical intervention.

Between two data centres, lets assume I did 6 visits a year, and that we had one ~30/min incident a month at $50/incident (it was less, but I don't remember the exact details, and it doesn't matter for this exercise). Let's assume I lost a whole day every visit (I didn't, though it got close at times), and "charge" $1000/day for my visits. That adds up to $6k/year for my time, and $600/year for remote hands, or ~$110/year per server. For comparison the colocation cost us ~$17k/year, or ~$283/year per server. These costs were pretty stable by number of servers, and so favored using fewer, more powerful servers than we might have otherwise.

So that added the cost of renting space at a manned colo facility instead of having the servers in the office (we did have a rack of servers that didn't need 24/7 attention at our office as well).

The rest of my time was spent on devops work that in my experience tends to be more expensive (on the basis of having contracted to do this kind of work on AWS too, and know the difference in billable hours I'd typically get per instance on AWS vs. per physical server on colocated setups) on cloud setups because complexity tends to be higher.

[erik_seaberg]

> costs were pretty stable by number of servers, and so favored using fewer, more powerful servers

Didn't this make each failure a bigger hit to your overall capacity? How much redundancy did you have? I used to work in adtech with colocated hardware, and it was old and failed a lot, but they had enough it didn't matter ("we're down 5/120 in Germany but we can swap them out while we're there next month").

[vidarh]

In that case it was ~60 servers, so losing any single server made little difference, but yes it of course needs to be a consideration if your number of servers is low enough.

It was also a fully virtualised setup that could also tie in rented dedicated servers or cloud instances via VPNs as needed. So where it made sense or if we had an urgent need, we had the ability to spin things up as needed.

E.g. we had racks in London, but rented servers at Hetzner in Germany (Hetzner got close to the cost of the colocated servers, though mostly because rack space in London is ridiculously expensive; it might actually have saved us money to put servers in their colo facilities in Germany, even with the cost of travel to/from them occasionally)

[lunchables]

This sounds like a great use case for AWS -- as cheap insurance. Just setup a VPC and VPN and if you have hardware fail just spin up an instance in AWS until you can replace your physical hardware. Pay $40/mo or so to keep the VPN active.