Hacker News new | ask | show | jobs
by dijit 1546 days ago
That’s not true in my experience. The cost of running on public cloud is extremely prohibitive.

There are some things that can run on cloud, but the heavy hitting game servers should really be bare metal for the baseline.

I gave a talk about this in Stockholm once, unfortunately it wasn’t recorded but I can share the slides if you want.
1 comments
imbriaco 1546 days ago
It's very true in my experience, and we've looked pretty hard at all the options. Thanks for the offer of the slides, I have first hand experience with this at scale.

It certainly depends a lot on how sophisticated your autoscaling is and how closely you're able to follow demand to limit waste, how well you can manage per-host utilization, whether you are CPU or memory bound, and lots of other factors. But even at truly massive scale the cloud hosting option can be very competitive without nearly as much management overhead.

link
dijit 1545 days ago
Interesting! I have the opposite experience and I also operated at quite a decent scale.

I wonder why our experiences are so different.

For context I was running 2,500+ instances at peak with 40vCPU and 256GiB of memory: the most expensive of those of course being in the regions with low density of players like South America and Australia.

We also had a predictive auto-scaler for our cloud components, I would estimate that our waste was 15% at any time, but if we ran bare metal only it would have been cheaper (except for the operational cost and the fact we needed lengthy commitments for hardware)

link
imbriaco 1545 days ago
We run rather a lot more than 2500 instances at peak and a variety of instance types dependent on game mode. Our waste is on the order of 5-7% depending on the time of day -- we can run less waste on the downward slope of the demand curve for instance.

We're also seeing some pretty compelling results from ARM based instances: https://www.wired.com/sponsored/story/changing-the-game/

link
dijit 1545 days ago
Aha, Fortnite. You guys will get discounts that we will never get, please keep that in mind.

Our waste was the inverse of yours, less waste on the upswing and more waste on the downswing, due to the long lived nature of our instances.

One instance takes roughly 1,000 players, but if you as a player decide to just keep playing, then we won't kick you out of the game for some hours. Some percentage of players will just continue playing as long as possible without matchmaking or map transfers.

link