Hacker News new | ask | show | jobs
by kasey_junk 915 days ago
I’ve looked extensively at the documentation for gke autopilot (a system I use extensively) and haven’t found any documentation on how they orchestrate those clusters under the covers.

I’ve always assumed it was a plugin to borg, not a different fleet orchestrator. Not to be overtly needy, but do you have a link to docs that contradicts that?
2 comments
dilyevsky 915 days ago
It is definitely not a plugin for borg. Borg is totally different api (source - i was borg sre). Afaik it’s actually vanilla k8s apiserver with some shimmed bespoke storage but it’s not really documented anywhere. You can test that fact using kubectl proxy though
link
verdverm 915 days ago
- https://cloud.google.com/kubernetes-engine/docs/concepts/clu...

- https://cloud.google.com/kubernetes-engine/docs/concepts/kub...

Each of these has more peer pages that are pretty good too

link
kasey_junk 915 days ago
You’ll note those documents are extremely careful in describing the control plane architecture to not promise you are running a stock k8s install. Which is why I’ve always assumed otherwise.

But I’ll trust the sibling comment which suggests the only bespoke component in gke is storage well enough to leave it alone.

link
verdverm 915 days ago
Maybe you are reading it differently than me, but when they refer to the same binaries, as if I managed it myself, as being the pieces they use, it definitely seems like the open source project being used.

> The control plane is the unified endpoint for your cluster. You interact with the control plane through Kubernetes API calls. The control plane runs the Kubernetes API server process (kube-apiserver) to handle API requests.

> A node runs the services necessary to support the containers that make up your cluster's workloads. These include the runtime and the Kubernetes node agent (kubelet)

link
kasey_junk 915 days ago
We are well into the reeds of what doesn’t matter, only in that fly has given us a very under the covers look at their implementation that is hard to find with other alternatives. But as someone who has run K8s in other contexts I find the following to be pretty circumspect (not in a way that causes me concern, I’m a happy gke user)

> GKE Autopilot manages the entire underlying infrastructure of clusters, including the control plane, nodes, and all system components. If you use GKE Standard mode, GKE manages the control plane and system components, and you manage the nodes.

There is a mile of implementation detail in that. Which I’m happy for them to keep on their side of the street.

link
verdverm 915 days ago
> I’m a happy gke user && Which I’m happy for them to keep on their side of the street.

100% agreement

iirc, there was a time where I thought they were doing some consolidation things with how they run the control plane, then at some point my cluster updates had a warning related to control plane unavailability during an update, this was on a single node cluster

I get what you are saying though, there's probably some magic going on somewhere, but after many years on GKE, I don't really think about it.

link