[eyberg]

1) VMs have hardware backed isolation - containers do not.

2) Containers share the guest kernel. To elaborate many/most container users are already deployed on top of vms to begin with - even those in private cloud/private datacenters such as openstack will deploy on top since there is so much more existing software to manage them at scale.

3) Platforms like k8s extend the attack surface beyond one server. If you break out of a container you potentially have access to everything across the cluster (eg: many servers) vs breaking into a vm you just have the vm itself. While you might be inside a privileged network and you might get lucky by finding some db creds or something inside a conf file generally speaking you have more work ahead of you to own more hosts.

4) While there are vm escapes they are incredibly rare compared to container breakouts. Put it this way - the entire public cloud is built on virtual machines. If vm escapes were as prevalent as container escapes no one would be using AWS at all.

[skuenzer]

I agree, an argument for 4 is the fact that the hypervisor attack surface can be scaled up and down by adding/removing virtual devices. There is only a little set that stays permanently, like 30+ hypercalls on Xen. Overall compared to a standard OS interface (Linux has in the range of 350+ syscalls) this is still very little. The Solo5 VMM project tried even out another extreme by reducing the hypercalls to less than 10 if I remember correctly.

[felipehuici]

It's also worth mentioning that a hypervisor's API, like Xen's, is much more stable; the Linux one is constantly growing.

[skuenzer]

Very true. And we also did not speak about the heavily multiplexed system calls like `ioctl`.

[MaxBarraclough]

> the entire public cloud is built on virtual machines

Some cloud providers will trust containers to isolate different customers' code running on a shared kernel, but it's not the norm. I think Heroku might be one such. There's at least one other provider too, but frustratingly I'm unable to recall the name edit found it, it was Joyent, who offer smartOS Zones. [0]

[0] https://news.ycombinator.com/item?id=25838037

[pjmlp]

Hence why everyone is now turning containers into micro-VMs, thus the sales pitch from containers is kind of waning.