[eblanshey]

Does anyone know how easy it is to set up autoscaling with Docker Swarm running on Google Cloud or AWS? We're looking to get starting with Docker Swarm or Kubernetes soon, and are considering using Docker Swarm because of its simplicity and developer familiarity with Docker Compose (we use it for our dev environment). We just want to add nodes to a cluster as traffic spikes and subsides.

[kingrolo]

Google Container Engine supports cluster autoscaling to automatically add nodes with load. It's listed as a beta feature though.

I've tried most of the Docker orchestration offerings and Container Engine seems by far the nicest. Swarm and Compose are really simple for getting up and running, but when we evaluated them there was still a missing piece required in that there was no neat way to do zero downtime deployments.

There's a tool called Kompose to convert docker-compose config to kubernetes manifests (https://github.com/kubernetes-incubator/kompose) although whilst it's nice to get you started we tend to maintain them separately now.

[KenCochrane]

Have you looked at Docker for AWS yet? https://www.docker.com/aws

It will setup your swarm, which uses auto scaling groups for the worker nodes. You can then configure the auto scaling groups how ever you want, to scale based on your cloudwatch metrics, etc.

There is also a Docker for GCP product in beta. https://beta.docker.com but I don't know how auto scaling works for it.

Disclaimer: I work at Docker on the Docker for AWS product.

[013a]

I think most people would suggest that, if your use case is at a stage where that is important to you, Swarm is not the right thing. Kubernetes or ECS are better choices.

[truetuna]

I wouldn't recommend ECS. I've used it for a little over 6 months and even for trivial things, it lacks. A couple examples that come to mind include not being able to pass host environment variables into your container instances easily, and not being able to specify that a service must run on all hosts.

Theres an open issue (made ~2 years ago) on GH for the 1st example and it still hasn't been resolved.

[hefeweizen]

In the context of Docker Swarm and Kubernetes, autoscaling refers to container level scaling ie. given a set of nodes, any autoscaling function would manage the number of containers that are currently running on these nodes.

For instance/node level autoscaling (which is closer to what you need), I would recommend using the autoscaling features provided by AWS/Google Cloud.

[eblanshey]

> I would recommend using the autoscaling features provided by AWS/Google Cloud

It would have to be integrated with Kubernetes though -- when we push a new docker container, the container would need to be updated on any new machines created. We'll look into GCP's autoscale solution.

[dalailambda]

The node level autoscaling doesn't need to be integrated with kubernetes, all it needs to do is create a new instance and register it as a node through normal channels.

Even if you don't need autoscaling, I'd suggest still using autoscaling groups and setting it to a fixed number of instances, so that instances will automatically get restarted if they go down.

[hefeweizen]

Yeah, any new machine instance has to join the Swarm (and its equivalent in kubernetes-speak). But that can be decoupled from kubernetes or docker swarm mode.

As for image management, it would depend on how you would like to propagate new images. With a private docker registry, you could potentially point each new instance to the registry and take care of propagating new images. I favor this approach since it keeps everything separate and easier to manage.

[acejam]

I suggest looking into Amazon ECS. They have auto scaling features that can trigger based on container-level alerts and thresholds.