[arghnoname]

CPU time is also more complicated to schedule in VMs.

For instance, the guest operations generally assume they're running on physical hardware and use spinlocks for some small critical sections. Under a physical hardware assumption this can be the correct approach, because the contending thread will leave the critical section soon and this overall performs better.

However, if the contending thread's vCPU is scheduled away by the hypervisor, the other thread may spinlock until the other vCPU gets scheduled back in. This wastes cycles.

A single operating system that uses OS-level virtualization (i.e., containers) has a more complete view of the system and can better multiplex the existing resources. That said, OS-level virtualization is generally accepted to have less isolation than VM isolation and solving the VM problems with, for instance, spinlocks might be easier than solving the isolation problems with containers, which is near intractable given the size of the kernel.

Unikernels try to take this approach and have a lot of the benefits of containers. If you squint, what we're really looking at with unikernels is a microkernel that uses virtualization support in hardware for robust process isolation. What's interesting to me is the question that I never see asked, which is whether we should revisit the microkernel architectures instead of laying more crap on top of monolithic kernels. The problems with Mach in terms of IPC time have been largely mitigated/eliminated with the L4 branch of microkernel.