[nix0n]

> that server is almost certainly going to be a VM and not bare metal

I understand that this is normal but I've never understood it.

If all the containers are running the same company's applications (so they don't care about security boundaries between them), what's the difference between having all the containers under the same kernel vs separate kernels?

[donavanm]

The VM layer gives you an aspect of fungibility that commodity hardware doesn’t. It’s being able to over provision, dynamically reallocate hardware resources, or do things like live migration and entire system snapshots. That hardware/system management aspect is what VM’s give you and containers don’t.

Note: if you want to conflate “containers“ with an entire job management and scheduling system (“k8s”) then you’re not actually talking about the current target customer for VMware.

[sofixa]

> It’s being able to over provision, dynamically reallocate hardware resources, or do things like live migration and entire system snapshots. That hardware/system management aspect is what VM’s give you and containers don’t.

None of those matter in the slightest with containers. Why would you need to reallocate hardware resources when the containers can run on another piece of hardware? You would snapshot the relevant storage, not the whole OS and kitchen sink.

VMs as an intermediary between hardware and containers is just a waste of resources - both directly (RAM, CPU, storage to run a useless OS with no benefit) and indirectly (all of those VM's OS needs maintaining and patching).

It's basically a hold over from the olden days of "everything is a VM".

[stackskipton]

We run everything on VM just for flexibility. Want to stand up a new machine for testing? Boom, run the script, new Ubuntu server. We need to decommission a machine. Shutdown all the guests, move them over to new machine and start them.

Sure, most of what we do is very cattle and we could run on bare hardware but why not take advantage of easy to add flexibility.

[sofixa]

I'm not sure I follow. Ideally your bare metal machines have an easy bare OS with some config that is easy to repave, and are ephemeral-ish (not a lot of state).

Your "machine for testing" would be a container. Decommissioning a container is easy. Decommissioning a physical machine only happens when it's obsolete.

[stackskipton]

sigh Not everything in the world is containers. Applications running outside of container still exist in many companies for various reasons including political ones.

[jamesfinlayson]

Yeah this - I know my container is pushing a lot of things to containers (I think customer-facing web stuff is a reasonable use for containers) but stuff like ActiveDirectory controllers, Exchange servers - pretty much any Microsoft thing running on Windows - will continue to run on a VM for the forseable future.

[sofixa]

Okay, but that's not at all what this thread is about? We're talking about having a VM intermediary between containers and bare metal. Not applicable to cases where you don't have containers because your workloads aren't fit for them.

[wmf]

It's cargo culting. Even 37 Signals fell for it.