[dijit]

Except 2 things:

1. Kubernetes manifests require "requests" to be specified. (mem/CPU allocation)

2. Getting 100 VMs identical is not difficult on the cloud.

The point I'm making is that you've already abstracted a lot of the things away with Cloud, and we abstract the same exact things even more on top of kubernetes.

If K8S was running on bare metal I'd agree with you though.

[orf]

If you can’t understand why it’s more expensive and less efficient to run 1 unit of work per server on 100 servers as opposed to fitting them into 20 larger servers then I’m not sure what to say.

[dijit]

I'm not sure why we're shifting around so much, I never claimed "efficiency" and especially not "efficiency of micro workloads".

This whole thread is discussing the "mental overhead" of managing VMs vs Kubernetes.

if you have to define the "size" of your workload, it hardly matters if it's a VM or k8s. You need to define the size.

Kubernetes can be more fine-grained (I want 1/4th of a CPU!) but you still define it.

I'm not talking about cost, or really anything, only that the original claim I originally responded to: "hurr durr but with VMs I have to configure everything!" but that is the same on kubernetes.

VMs are already a pretty good abstraction if you're looking at carving up compute resources. My "frustration" if I even have one is that we are doing both, one on top of the other. Which feels extremely wasteful.

But like everything it depends on your workloads, and I'm used to having things that consume entire CPU cores, not 1/4th of one. (I'm also not used to making web services these days, and kubernetes is optimised primarily for that kind of stateless workload)

[orf]

I'm also used to having workloads that consume entire CPU cores, and as such I'd like the number of CPU cores dedicated to log aggregation, system monitoring, metrics etc to be as reduced as possible. I'd also like to not spin up a bunch of new VMs to do a rollout, and I'd also like to run all those small satellite workloads that always appear on the same platform. Oh, and I'd not like to have to run something that needs 3gb of memory and 3 cores on a machine with 4gb of memory and 4 cores because I'm constrained by AWS instance sizes.

Mixed workloads on smaller, larger machines are great for this.

With VMs you do need to configure everything, compared to a baseline stripped down AMI/image that runs nothing but docker and a Kubernetes daemon.

Yes, you can enumate Kubernetes with a bunch of custom tooling. No, it's not better. Yes, it is harder.