Hacker News new | ask | show | jobs
by amannm 1545 days ago
Don't game servers need to run on bare metal to minimize latency?
6 comments

This is my field, so even though I’m literally titled “expert”[0]- take what I say with a pinch of salt.

When it comes to gameserver performance what tends to be the biggest issue is predictable performance. So, kubernetes is not inherently antithetical to running game servers as long as you don’t share the cpu core(s) with anything.

In fact there is a project that (while I was not a part of) was spearheaded by ubisoft in collaboration with Google for running game servers on Kubernetes (called Agones)[1].

Ubisoft also used “Thunderhead” (which this project seems to be a variant of) on Azure with rainbow 6 siege. So it’s definitely not considered a show-stopping problem.

When it comes to containers/VMs the biggest hit in terms of performance is in this order: disk/network/memory access/CPU ;; when talking about game servers the most important things go in the reverse of that order, which means that the biggest performance losses are not as important. Network latency might seem like a big deal but you’re limited to frame times in the best case and geographical differentiations will eat much more than the combined weight of frames and Kubernetes networking.

Agones itself bypasses a lot of the Kubernetes networking latency additions, though, not all.

[0]: https://www.linkedin.com/in/jharasym

[1]: https://agones.dev/site/

The Siege team also did a pretty interesting talk about running game servers in the cloud and the trade offs : https://www.gdcvault.com/browse/gdc-17/play/1024036

It’s quite old now but interesting for anyone whose experience is purely running on their own hardware.

Not necessarily in my experience! Most games don't hit the CPU particularly hard.

Mostly reliant on (hardware offloaded) network latency, memory, and storage. Consistency being the thing of most importance. A spike is felt - interpolation can only go so far

CPU virtualization extensions make the latency cost of VMs hardly noticeable, containers are technically better but it's a split hair.

On an unloaded host and a single VM you can achieve essentially identical performance as bare metal - the CPU extensions, pinning cores, and huge pages are key

Some performance metrics even improve! Disk operations tend to do better from an added layer of memory involvement

Edit: there are options for multi-tenant fairness whichever way you go -- Bare metal, VMs, or containers

Performance isn't much of a worry, more so the 'handles'

How important is GPU access for game servers?
Not important at all if the game server is properly isolated from the game client.
Very little, most of them run headless with no graphics

There are exceptions. For years you could only run Satisfactory servers by the server acting like a client - rendering and all

Using container technology doesn't necessarily mean adding virtual network devices/proxying/forwarding. In practical terms, you can run in "host" networking mode (no separate network namespace) on Docker or Kubernetes, and most people would still consider those "containers".

In this case they seem to be NAT'ing packets from a host port on the correct Kubernetes node (the one running the container) to a port in the container, which can be done fast enough with iptables (or similar mechanism used by Kubernetes).

I think keeping that kind of stuff to the minimum would be important.

iptables is a good example -- it can scale rather poorly! Packets are run across the chains at length until a matching rule is found.

For most configurations this isn't a problem - the rules are filtered against quickly.

If density reaches the point to where you have thousands of forwards, it'll slow down a lot!

You'll want to look into optimizations (eg: ipsets), offloading to hardware, or simply going to host networking

If you're running a handful of game servers on each machine, each with a single NATed port, you're fine. At bigger scales it's a problem, but as you mention there are better solutions now.
Not really. Minecraft server notoriously ran in JVM for most of it's existence. Container virtualization is unlikely to add more than a few microseconds of latency.
> Minecraft server notoriously ran in JVM for most of it's existence.

Alas, it still does; at least for the Java client.

Nothing about kubernetes implies virtualization. People only use VMs because they're easier to provision. Technically you could provision physical servers with kubernetes and run containers on them with zero overhead.

Or perhaps more realistic you could provision a tier of very low latency VMs.

Not really. The major concern is getting clients to one machine and running a stateful connection so you don't need to pass messages through some kind of slow, multi-hop broadcast system. The CPU overhead of containers or vms is pretty minimal compared to this main concern.