[zinodaur]

I guess I would prefer "kubernetes but with VMs instead of containers". The overhead of running in a VM is not very high, and a hypervisor can restrict resource usage much more effectively - so that we could still bin pack map reduce jobs on the same machines as live site services

[ianburrell]

Kubernetes has pluggable container runtimes. There are ones for running VMs, including the new lightweight VMs. They use standard OCI images.

Using VMs with Kubernetes only makes sense when you need the strict isolation. If you are running own code, then containers are faster. Containers also perform better because they can share resources on host. In Kubernetes, containers can have minimum and maximum limits, which means they can dynamically use space not used by other containers. VMs need to be allocated memory when they start.

[pjmlp]

So does plain Docker runtimes nowadays, specially since the company had to pivot into something else.

[kube-system]

If your kubernetes nodes are VMs then you can do both at the same time for different parts of your application.