[iopq]

Yeah, but then there's still the issue of secrets, you need to have testing PayPal credentials, testing mailing service credentials, etc. There's the issue of deploying changes fast without leaving files in an inconsistent state (you don't want half of some file to run). How about installing the required dependencies?

I don't use Docker, but those are problems I can think of off the top of my head.

[eropple]

Docker doesn't credibly solve the credentials problem and the other problems you outline (which do exist) are as practically solved with something like Packer. And I mean, I'm not a Packer fan--oh look, VirtualBox failed to remove a port mapping for the VM that just shut down, throw away the whole build--but it's built on much, much more battle-tested technology with a much wider base of understanding.

(And, later, if you want to play with Docker, Packer lets you do that too. But you should use the Racker DSL in any case, because life is too short to deal with Packer's weird JSON by hand.)

[chucky_z]

Thanks for pointing me to Racker (https://github.com/aspring/racker). I'm currently building Packer and Terraform images with chunked together Python scripts that work, but I wouldn't call them a great solution. I'm actually using Packer specifically so I can start with regular EC2, and then move to a more Docker-based infrastructure.

[eropple]

Packer severely frustrates me, with the maddening regularity in which it fails just for funsies. Or the consistent but completely inane ways that it fails, like refusing to proceed based on not finding a builder for an 'only' or 'except' clause (making it nearly impossible to re-use provisioners and post-provisioners across multiple projects). Racker does help--my shared Racker scripts are in a Ruby gem--though I think that it pretty much removes Packer as anything more than a dummy solution into which you dump directives on a per-builder basis. As a tool that you carefully feed the bare minimum of information to do its job in any specific situation, though, it works okay.

Terraform, on the other hand, I think is a huge, huge mess, and I don't think they're going to fix it. I wrote a Ruby DSL for it the last time I tried to use it in anger, only to encounter that Terraform didn't honor its own promises around the config language it insisted on instead of YAML or a full-featured DSL of its own. Current client uses it, and every point release adds new and exciting bugs and regressions in stuff that should be caught by the most trivial of QA. For AWS, I strongly recommend my friend Sean's Cfer[1] as a better solution; CloudFormation's kind of gross, but Cfer helps.

[1] - https://github.com/seanedwards/cfer

[acdha]

Credentials have to be managed separately from Docker anyway.

> There's the issue of deploying changes fast without leaving files in an inconsistent state (you don't want half of some file to run). How about installing the required dependencies?

rpm / dpkg also install dependencies, are quite fast and well tested. They have the advantage of working in a standard environment which most sysadmins know but the disadvantage that you need to configure your apps to follow something like LSB (e.g. install to standard extension locations rather than overwriting system files, etc.).

The one issue everything has is handling replacement of a running service and that's not something which Docker itself solves – either way you need some higher level orchestration system, request routers, etc. Some of those systems assume Docker but that's not really the value for this issue.

[regularfry]

> the disadvantage that you need to configure your apps to follow something like LSB (e.g. install to standard extension locations rather than overwriting system files, etc.).

Common misconception. You only need to do this if you're going to try to push the packages upstream. If they're for your own consumption, you can do what you like. Slap a bunch of files in /opt, and be done with it - let apt manage versions for you and be happy.

As with many things, this is one area where you've just got to know what to ignore. It's simpler than it looks.

[acdha]

I think we're actually talking about the same thing – I said “like LSB” simply to denote following some sort of consistent pattern, which will vary depending on how widely things are shared.

/opt is defined in FHS for local system administrator use, so installing your company's packages there is actually the recommended way to avoid conflict with any LSB-compliant distribution as long as you use /opt/<appname> instead of installing directly into the top-level /opt:

http://www.pathname.com/fhs/pub/fhs-2.3.html#OPTADDONAPPLICA...

[mentat]

rpm and fast don't really go together. dpkg is much better. dnf will be interesting.

[maratd]

> Yeah, but then there's still the issue of secrets

How would Docker help with this? Genuinely curious.

I store them in bash scripts outside the repo that populate the relevant data into environment variables and execute the code. The code then references the environment variables.

> How about installing the required dependencies?

There are two kinds. On the OS level and on the platform level.

On the OS level, you can have a simple bash script. If you need something more complex, there are things like Chef/Puppet/etc.

On the platform level, you have NPM/Composer/PIP/etc which you can trigger with a simple cron script or with a git hook.

> There's the issue of deploying changes fast without leaving files in an inconsistent state

So the argument here is that you're replacing one file in one go vs possibly thousands? That in the latter scenario the user might hit code while it's in the process of being updated?

Ok. With docker, you would shut it down to update. You would have to.

Same goes for the traditional deployment? Shut it down, update, start it back up?

You can, of course, automate all of this with web hooks on Github/Bitbucket, for both docker and the traditional deployment.

The traditional deployment should also be faster, since it's an incremental compressed update being done through git.

[robszumski]

Kubernetes secrets are a really great solution to this problem. [1] They are stored at the cluster level and injected into the pod (group of containers deployed together) via a file system mount. This means that each pod only has access to its secrets which is enforced by the the file system namespace. If an entire machine is compromised, only the secrets of pods currently scheduled onto that machine are able to be stolen. That's a high level, but it's worth taking a look at the design doc.

Edit: forgot to mention, the file system mount means that they don't need to be in env var, which are fairly easy to dump if you have access to the box or are shipping containers around in plain text.

1. https://github.com/GoogleCloudPlatform/kubernetes/blob/maste...

[iopq]

I don't know if Docker helps with this, I don't use Docker. But some kind of solution has to exist.

How AWS does updates is it first downloads the new code into a separate folder and then switches the link to point to the new folder instead.

But AWS has an unsatisfactory feeling because it downloads the entire code instead of doing a git update. These are all issues that could be fixed, and someone has to do them. I have no idea if Docker helps with any of them, but the opportunity is still there.