[charcircuit]

I think part of the confusion is that server can refer to both a physical box and a VM. Abstracting away servers is about not having to think about VMs. You have a service which requires some amount of compute, memory, and storage and you just want that service to run. There is value to not having to worry about provisioning a VM or administering it.

[jasonrichardsmi]

But you still have to do that in Kubernetes, unless you are running Fargate. Someone has to provision and maintain that machine, and in the process introduce a ton of administrative overhead.

[danpalmer]

That's true, but not a particularly interesting fact.

For example, if you use GKE (Google Cloud's K8s offering), you attach your K8s cluster to an auto-scaling node pool and it handles (de)provisioning of your VMs for you. You essentially don't care about the VMs, there's essentially no overhead.

If you are in a private cloud, this also creates a good "API boundary" between the team responsible for running hardware, and the team responsible for shipping software to run on that hardware. On the former side you can essentially just adopt a machine into the cluster and leave it, and on the latter side K8s lets you programmatically reference resources, but you don't need to know how/where they came from.

[jobegrabber]

I would say it is an interesting fact, since it's this "good 'API boundary'" that, as you said, enables one to separate concerns, be it between different teams in an organisation or between a service provider and its users.

Yes, you don't need Kubernetes to come up with your own implicit or explicit API boundaries, and these might not be needed for smaller projects. I agree that Kubernetes is often used where it's not strictly needed.

There are things which strict abstraction, and with it, separation of concerns provide. The crucial point is that certain things are enforced.

[danpalmer]

Sorry, I meant not an interesting fact in public clouds where they take care of all the VMs for you, even without Fargate.

You're right though, the abstraction layer is very interesting for _enabling_ public clouds, and for private cloud team ownership.

[Eritico]

Its not the same. You can easily run node pools automatically because your abstraction layer is k8s with containerd or docker.

You also know that you can throw away VMs because they don't contain any state. You are not losing data just because you kill a VM or a VM breaks.

It is way easier to just spin up n nodes and provision them all equally than whatever you did before.

In my team, we can manage way way way more nodes than we ever could. We spin up 100 VMs and destroy them on a regular basis automatically.

Gardener for example supports autoscaling on bare metal. The whole ecosystem is providing tons of great options.

[dijonman2]

I think there is a healthy dose of anti-vm bias in your viewpoints.

Lauding Kubernetes because ops work took too much of your time is just shifting your burden elsewhere, even if that means paying a bit more for an offering like ECS fargate.

Any environment with configuration management can treat instances as ephemeral. It’s a best practice.

I view docker more as a package manager, no more dependency hell.

In any event K8s is sprawling, it will soon be too complex for its own good. Assuming it’s not already.

[Eritico]

I only answered the question why it is different with kubernetes.

I dont have anything against VMs. Feel free to click yourself a VM on any cloud provider, use it however you like.

K8s abstracts VMs away and i have and had real issues with maintaining VMs. Docker filling up the node with logs. Unable to upgrade the BaseOS due to python dependencies. Managing the same VM stack through ansible and everything ansible or chef brings to the table.

There have been plenty of self healing mechanism in place which do solve unfortunate issues. Memory? The service restarted, was offline for 3 minutes and is now working again. Node disk full? Pods get scheduled away, new node comes up, done. Update/upgrade of nodes? Nodepool does it for me.

For me, k8s has 2 real issues like memory (swap support is wip finally!) and stateful workloads like a database. But the concept of an operator shows a bright future.

k8s also does one thing very nice: It enforces certain aspects which are a pain in the ass later. That VM which wasn't updated for years and run just fine? Now there is an issue and it needs to be fixed asap. But now the debian repositories are no longer available. I have to fix apt srce list first, then i need to fix dependencies and then i need to restart it.

[gizzlon]

That's a PAAS and they existed long before k8

[danpalmer]

A good mental model is to think of K8s as a portable PaaS with a more well-defined API. That's a good thing, not a criticism of K8s.