| "Misconception: You should have only one process per Docker container!" as soon as you start treating docker images as anything other that isolated statically compiled executables, you're not going to get the best out of docker. if you are bundling inits, crons and companion apps into a single container then you need to stop, go back and either re-factor your code, or go to Full on VMs, why? because the networking is terrible. There are three great advantages to using real VMs over containers: o Networking o Isolation o hot migration and resource allocation Networking: every instance of a service can have its own IP, and can be trivially tied to DNS automatically. scoped service discovery that's only sortof just possible now. however it uses immature tools with limited professional experience to back them up. DNS, DHCP with subdomains means images can be dropped in without any hard work Isolation: Its far harder to break out of a VM than it is a container. Especially if you are dealing with persistent storage and need to allow a container to write outside of its own chroot. Hot migration: This is killer. Hardware fails. having a cluster that automatically migrates around contention and hardware failure, without the app having to worry is worth many thousands of man hours. Yes making your own clustering system is fun, but its really quite hard to do well. Why bother when the hypervisor can do it for you? There are three things going for docker: Configuration library: There is a rich library of prebuilt images Baked in fudges: You can bake in your dirty hack into the container, so long as you script it into your build job, its repeatable. Speed: yes there is less overhead. but lets be honest, how often have you hit up against VM speed issues that were down to your machine using too much CPU/memory? (if you're on AWS, no, you've not. AWS is dogshit slow, and expensive.) Everything else, like immutable builds, easy dev environments et al, can be achieved already, and without much work. |
- Creating a scratch VM (not even provisioning) is a speed bump when you want to re-running scripts on a pristine environment to validate them. Starting a Docker image takes ~1s.
- Provisioning is slow! Reddit suggests installing a plugin (vagrant-cachier) to keep you sane. I ended up downloading a plugin to take VirtualBox snapshots of my VM and even that was depressingly slow. Docker commit takes maybe 20s on a big layer.
- VirtualBox shared folders are pretty bad, so apparently I should install vagrant-bindfs (and NFS packages). -v is effectively zero effort.
- I had to keep on shutting down and starting up the VM to tweak memory settings - too big and my laptop died, too small and nothing would start. You get far more flexible controls in Docker.
I couldn't help but think all the way through that putting everything in a Docker container would just be a far nicer experience, even if it does violate the single process rule - for this use case, Docker solves some papercuts very effectively.