Hacker News new | ask | show | jobs
by NightMKoder 1404 days ago
This may not be a surprise to some, but when folks talk about reliability of the control plane, they usually think failure means their web service goes down. That’s not true. If you shot the kubernetes control plane, the individual servers can’t talk to it anymore - so they do nothing. Docker images that were running stay running. They even get restarted if they crash (via restartPolicy). Services that had specific other pods they were referencing continue referencing those pods. In net: everything except for kubectl and other kubenetes internals keeps working.

That said, one piece that isn’t talked about frequently is the network overlay. Kubernetes virtualizes IPs (so each pod gets an IP), which is awesome to work with when it works. But if your overlay network goes down - god help you. DNS failures are the first to show up, but it’s all downhill from there. Most overlays take great care to degrade well, so they’re not tied to the control plane, but I have yet to find one that’s perfect. The overlay is the part of kube that truly isn’t failure tolerant in my experience.
1 comments
411111111111111 1404 days ago
> Kubernetes virtualizes IPs (so each pod gets an IP), which is awesome to work with when it works

Kubernetes does no such thing.

Weave Net, which is likely the most used CNI, does. There are other options however, and some of them use baremetal routers via bridging or even VLANs for example.

https://kubernetes.io/docs/concepts/cluster-administration/n...

link
simiones 1403 days ago
The fact that each pod has an IP is a core assumption of Kubernetes. Sure, the CNIs are responsible for actually implementing this, but it is a required part of their contract to provide 1 unique IP per pod (or, more precisely, either 1 IPv4 or 1 IPv6 or both per virtual NIC per pod - to cover dual-stack support in 1.24+ and Multus).
link
411111111111111 1403 days ago
That's probably true, but also irrelevant to the question wherever kubernetes virtualizes IPs. But now that I'm rereading my comment: it does look as if I'm also talking about each pod having one IP. That was bad quoting / phrasing on my part, as I wasn't contesting that at all.

With flannel you could provision the IP through DHCP by bridging the network adapter of the pod to the physical interface to get an IP from a router appliance for example.

It's probably also possible to dedicate actual network adapters to the pod, but I've never attempted that... And that obviously wouldn't scale as it's hardware

link
simiones 1403 days ago
Oh, you were focusing on the explicit notion of virtualizing IPs. I thought you were pointing out that Kubernetes itself is not the one generating the IPs, since it's the CNIs that do so, which are not built-in...

Either way, we are in agreement I believe. Kubernetes mandates for CNIs that they must allocate unique IPs, but they do so through a variety of mechanisms, sometimes even using external infrastructure.

link