[Rillen]

Making pods mutable would break the core benefit of what kubernetes imperial system does for you.

This GitOps 'nonsense' gives me a well defined and automatically backuped infrastructure setup with audit build in. It doesn't allow someone to snowflake around which is brilliant and forces you and your colleges to manifest stuff and not forgetting it and degrading your system over time (nix and nixos are also great examples of such systems)

This reminds me of the time i learned html and wanted to set new lines all over a text instead of using proper paragraphs and letting html take care of the proper formatting.

I would like to see a better/stronger statefulset though. As long as this pod is alive, make sure its state is not interrupted. Like allow a pod to be migrated to another node.

Nonetheless, i'm in the middle of setting up kubernetes with kubeadm and cilium network. Its already really easy to do so. It will just get more easy and more stable over time and its already great.

When you look at the storage example: yes its more difficult then just using a hard drive. But you ignore the issue with one hard drive: Backup, checksum / bit rot and recovery. With a storage layer, you can actually increase replica count, you can backup ALL storage volumnes automatically.

the same with networking: With cilium you can now have a lightweight firewall with dns support.

It is much more critical for the whole industry to start rebuilding software to be more cloud/container native. This will reduce the pain points we have right now and will make it more resilient to operate. For example Jenkins: Instead of one big master, have a ha setup for your working queue, a pod for a dashboard and schedule workers on demand.

My personal conclusion: Don't use it, if you don't need it. If you need it, embrace the advantages.

[throwaway894345]

> Making pods mutable would break the core benefit of what kubernetes imperial system does for you.

Which core benefit is that? I’m not following.

> This GitOps 'nonsense' gives me a well defined and automatically backuped infrastructure setup with audit build in. It doesn't allow someone to snowflake around which is brilliant and forces you and your colleges to manifest stuff and not forgetting it and degrading your system over time (nix and nixos are also great examples of such systems)

TFA says you can still use the GitOps “nonsense” if you want under his proposal.

[lazyresearcher]

Can't you use node affinity to stop k8s from moving a pod to a different node whilst alive?

[Rillen]

Thats not the problem this would solve. As long as the node runs and the pod itself runs and there is no issue with a pod with a higher priority, k8s will not throw it from that node.

But imagine a database as a pod with 60gig of ram and a ha setup. Now you need to update your node, what does k8s? It will throw it out and creates a new one which needs to recover or read all the logs to fill up 60gig of ram again from nothing. Instead it could migrate this pod to another node and keep the downtime to a minimum.

Or a jenkins master, it has to shutdown on node 1, recreate to node 2 which takes time and then your agents need to be able to recover from it.

You have to be able to roll through your whole k8s infrastructure to update every node on a regular basis; Alone for security reasons.

[joana035]

Sooner than later kubernetes will support live migration of workloads via checkpoint-restore of processes, like xen, and many other software already has.

https://en.m.wikipedia.org/wiki/CRIU

EDIT: https://github.com/kubernetes/kubernetes/issues/3949