[__turbobrew__]

You can get a poor mans multi topology (dc, region, az, rack, etc) system by splitting a kubernetes control plane and etcd cluster across that topology. For example if you have 5 dcs you can run a control plane and etcd instance in each dc and be impervious to 2/5 dcs failing.

The performance of running etcd clusters across high latency links (100-200ms) is not as bad as I would have originally thought and scales into tens of thousands of pods without too much trouble.

As long as you label kubelets with topology information, deploying selectively in different topologies is as simple as specifying the correct node selector. Workload high availability can be ensured with topologyspread to make sure not too many pods of a single workload are running in the same failure zone.

If you can go k8s native with your infra this stuff becomes easier, but by no means easy.