The main resource issue is because Docker runs in a VM, it can't share memory with the OS. So it's using the configured amount of memory (2GB by default) whether you're running busybox or an entire Kubernetes cluster.
I think the Virtualization Framework mitigates that to some extent. The way Docker was required to run on x86 it needed a Linux VM which I believe the Docker images ran atop. Under the new model, I think they run on bare metal and share memory.
Regardless, for me the cost of adding enough RAM to support Docker is less than the headache and expense of spinning up Docker images in the cloud.
M1 Docker still needs a Linux VM, Virtualization Framework is just a different way of doing that. I'm not aware of any VM that can share memory with the host, but maybe Apple has some black magic up their sleeve.
Yeah, fundamentally Docker is Linux on Linux. If you are running Linux on MacOS, it's not Docker, it's a VM. So Docker installs are Linux Docker Image -> Linux VM -> MacOS Virtualization Framework.
For that reason alone, I suspect Linux will always be the best host system for Docker images.
Seems like Apple's Virtualization Framework supports some sort of memory ballooning mechanism (https://developer.apple.com/documentation/virtualization/mem...), which could hopefully enables in the near future dynamic memory support like Docker Desktop on WSL 2 is able to do (if it's not already the case, I haven't tried the M1 preview).
Regardless, for me the cost of adding enough RAM to support Docker is less than the headache and expense of spinning up Docker images in the cloud.