[omginternets]

>It's typical in a VM farm to have hosts with a hundred VMs.

Several things:

1. We may have a different definition of "commodity hardware", but you're missing the broader point.

2. The broader point is that VMs are significantly less resource-efficient.

3. 1 & 2 notwithstanding, you're conveniently ignoring the issue of (re)start time

4. It's fine to use VMs, but it's frankly bizarre to fight tooth-and-nail over the ridiculous notion that they should always be preferred over containers.

[user5994461]

I am simply addressing the fact that it's perfectly fine and common to have hosts with a hundred VMs and it works flawlessly.

VMs are memory intensive because they duplicate the operating system. The starting point is around 500 MB per VM. That's the only meaningful difference in resources compared to containers.

I am not discussing that they have different starting and stopping time.

[arghnoname]

CPU time is also more complicated to schedule in VMs.

For instance, the guest operations generally assume they're running on physical hardware and use spinlocks for some small critical sections. Under a physical hardware assumption this can be the correct approach, because the contending thread will leave the critical section soon and this overall performs better.

However, if the contending thread's vCPU is scheduled away by the hypervisor, the other thread may spinlock until the other vCPU gets scheduled back in. This wastes cycles.

A single operating system that uses OS-level virtualization (i.e., containers) has a more complete view of the system and can better multiplex the existing resources. That said, OS-level virtualization is generally accepted to have less isolation than VM isolation and solving the VM problems with, for instance, spinlocks might be easier than solving the isolation problems with containers, which is near intractable given the size of the kernel.

Unikernels try to take this approach and have a lot of the benefits of containers. If you squint, what we're really looking at with unikernels is a microkernel that uses virtualization support in hardware for robust process isolation. What's interesting to me is the question that I never see asked, which is whether we should revisit the microkernel architectures instead of laying more crap on top of monolithic kernels. The problems with Mach in terms of IPC time have been largely mitigated/eliminated with the L4 branch of microkernel.

[omginternets]

>I am simply addressing the fact that it's perfectly fine and common to have hosts with a hundred VMs and it works flawlessly.

And that was never the point.

[user5994461]

Yet that was your conclusion.