[Eritico]

I can tell you what it does for me: Full disk on your VM? Nope. Storage is abstracted away. Your server will not fill up anymore. Only one service might break which might heal itself.

Is your VM/node broken? It will heal itself because you throw it away and the new VM/node is fixed.

It enforces the abstraction of Service and VM. You will not install normal software on that VM just because you can. You don't need to give access to a VM to a developer who then needs root access and has dependencies and doesn't update the VM.

You no longer have dependencies to your VM because you can't have dependencies on your VM OS.

Abstracting it away from your VM also streamlines things like logfiles. You no longer need to collect all logfiles from VMs because you do it for your services and for your services, you only do it once (if even, log to stdout and be done with it)

[goodpoint]

None of this is done by k8s. Network storage, hardware virtualization, network virtualization? The all existed before.

[withinboredom]

None of this is true.

[Eritico]

I'm describing my real life issues i have and had.

Feel free to actually write more than 'None of this is true.' in a way that a discussion is actually possible.

Tx :)

[withinboredom]

I run k8s on bare-metal, and I can say a full disk is certainly possible if you have a service logging a few mb/s. Things will break in fun and interesting ways, data will get irrecoverably corrupted, etc. Your entire cluster will probably even break if said node was the etcd leader. This is pretty easy to reproduce by simply saturating a network and then watching the etcd leader spill its guts in your logs once the network buffers fill up.

> You no longer have dependencies to your VM because you can't have dependencies on your VM OS.

Your containers rely on the OS's kernel and whatever features it was compiled with.

> You will not install normal software on that VM just because you can

If you're paying through the nose for managed k8s, this is true. If not, you'll eventually need to login to a node and diagnose some issue, which means installing things on the node.

> You no longer need to collect all logfiles from VMs because you do it for your services and for your services

Whatever you installed to collect logfiles is getting them from the VM's disk (in /var/log/pods in k3s), unless your container is redirecting them somewhere that isn't stdout.

[hodgesrm]

> If you're paying through the nose for managed k8s, this is true. If not, you'll eventually need to login to a node and diagnose some issue, which means installing things on the node.

Managed Kubernetes on Amazon (EKS) is quite inexpensive: $0.10/hr * 24 hrs/day * 30 days/month = $72/month. Other costs are VMs, networking, and storage, which you would have allocated anyway. There are some downsides like forced upgrades, but cost is not of them for our use cases.

We incidentally don't ever login to Kubernetes nodes using tools like ssh. It's asking for security trouble to have those ports open.

[orf]

It’s all true though.