[osipov]

There are many ways of using Docker and obviously different companies could come up with their own business cases for adopting the technology. So let me focus on one scenario and we can talk about whether it makes sense for your environment.

Software engineering is often difficult because programmers have to deal with inconsistent environments for development and for production execution of their products. Due to mismatches between these environments, developers often found bugs that surface in one environment but not in another.

Hardware based virtualization (VMWare, HyperV, etc) helped with the inconsistency issue because it enabled developers to create dev/test environments that could later be replicated into production. However this category of virtualization requires more computational resources (esp. storage) than operating system virtualization like LXC used by Docker.

In addition to requiring less resources than hardware virtualization, Docker defined a convenient container specification format (Dockerfile) and a way to share these specifications (DockerHub). When used together, these Docker technologies accelerate the process of defining a consistent environment for both development and production in your application. Dockerfiles are easy to maintain and help reduce the need for a dedicated operations team for your application. In buzzword speak, your team can become more "DevOps".

Docker, by the virtue of relying on a thinner virtualization layer than hardware hypervisors like VMWare, also has higher performance for I/O intensive operations. When used on bare metal hardware, you going to be able to get better performance for many databases in Docker containers compared to databases running in a virtual machine guest.

So to recap, Docker can help you

- maintain consistent dev/test/prod environments

- use less resources than hardware virtualization

- free up the time your team spends on dev to ops handoffs

- improve your app I/O performance compared to running in a hardware virtualization based virtual machine guest

However if you are using AWS, note that Docker Container Service available from Amazon actually doesn't give you Docker on bare metal. That's because Docker Containers run in AWS virtual machine guests virtualized by Xen hypervisor. So with AWS you are paying a penalty in resources and performance when using Docker.

[takeda]

Great, but what are the benefits of running Docker in AWS? You are still running VMs and you are being charged for running them. With Docker you are simply putting yet another layer of complexity, because now you have to run more beefier VMs, you now have problem with network communication between containers running on different hosts. So you will most likely need to use overlay network. You also decrease resiliency, because now when AWS terminated a single VM, all apps running on that node suddenly disappear.

I also don't get the argument about running the same container in dev/test/prod. For example my company is working on going Docker and one of the problem with these environments is that app running there has different configuration. So the idea to solve it is to create three different versions of the same container. Genius! But now are you really running the same thing in dev/test/prod? How is it different to what we did in the past? Especially that before Docker through our continuous delivery we actually were using exact same artifact on machines set up with chef that were configured the same way as in prod, while with Docker now we plan to use three different containers.

[osipov]

>what are the benefits of running Docker in AWS?

I don't see benefits to running Docker in AWS. In my opinion, AWS implemented its Docker-based Container Service very poorly. I advise my customers against using AWS when want to use Docker. There are many bare metal as a service providers out in the marketplace.

>the argument about running the same container in dev/test/prod

Is this issue really caused by Docker because you said that you had consistent environments when built by chef?

[nnain]

I installed the Amazon Linux Installation Image (AMI) with docker and Python 3.4 for a recent project. I know ElasticBeanstalk fairly well, but configuring this was a big headache. WASTED LOT OF TIME.

Instead of making life easy, it just added unnecessary burden of learning Docker for the future project members. Documentation is poor, had to hunt for hours for solutions to simple queries.

On AWS, I would suggest you stick with the basic Linux flavour that you know. Use their Docker build only IF you know docker very well.

[pavel_lishin]

> In my opinion, AWS implemented its Docker-based Container Service very poorly.

We looked into using it earlier this year.

The web UI was flat-out busted in several ways - they only listed the first 100 security groups, and we have a magnitude more than that. The command line interface was poorly documented, and was missing some of the functionality.

It was a total waste of a week; I wonder if they've fixed any of that.

[vosper]

Would you mind elaborating on some of the issues you see with Elastic Container Service? I'd hoped it would be something like an AWS-specific Mesos, but I haven't looked into it closely.

[notwedtm]

Can you name a few bare metal as a service providers?

[nickstinemates]

Rackspace, SoftLayer/Bluemix

[spdustin]

Your environment-specific config shouldn't be in the container, but described in the environment itself, whether through ENV vars exposed to the container, or a mounted volume of config files. This is a solved problem, in my (admittedly limited, compared to some other commenters) experience.

[bri3d]

Robust composition. Instead of needing a separate database, message queue, app server VM, you can run all 3 containers together on one omnibus machine, or separated as scalability needs demand.

Yes, you can achieve something similar using multiple layers in OpsWorks or different deployment schemes with normal Chef, but IMO containers make provisioning and deploying combinations of components easier than most other provisioning and deployment solutions. There's less opportunity for unexpected version collisions and because the network infrastructure is virtual, you can move containers between underlying VMs, allowing capacity planning without substantial reconfiguration.

[nogox]

Check out https://medium.com/@hyperhq/docker-hyper-and-the-end-of-gues...

I think this is the way Container-as-a-Service to go, instead of the current form of AWS ECS.

[stevelandiss]

@takeda, how do you create 3 different containers in Docker for dev/test/prod? What parameters are you changing in the three stages?

[hootener]

- maintain consistent dev/test/prod environments

We're pretty heavily invested in Docker. To this point, it's really nice knowing that all the developers are operating in consistent environments.

An additional bonus is that using Docker makes it really easy to propagate infrastructure changes to the rest of your team for use in development. And, more importantly, know those infrastructure changes are consistent across local dev setups.

As an example, I recently incorporated Sphinx search into a project we were working on. I didn't want to require devs to install Sphinx on their own machines and get it up and running for search to function properly. I also didn't want devs to have the overhead of running Sphinx for search on their local boxes unless they were actively working on something related to search. Basically, I wanted search to be optionally configured to run on startup.

I used a DockerFile to setup Sphinx in its own container, pushed the DockerFile to a Tools source repository we use, and then incorporated the build and running of that container into our startup scripts (just some simple orchestration stuff for the local machine written in bash). Now, if a dev wants a containerized search mechanism they run a simple bash script to build that container, then run another command to spin up our dockerized web app with a connection to a running sphinx search container. We do this for all of our services: mysql, redis, sphinx, the web app itself, and anything else we might need.

As an added bonus, all the Docker CLI work and orchestration of our application is easily hidden behind a shell script. If a dev wants to run the webapp, they simply run: app server dev. If they want the app with search they run: app server dev search from the command line. Developers never need to know what's going on under the hood to get their job done. From their perspective, it just works.