[graffitici]

Anybody has insights about using Docker Swarm? I imagine Kubernetes has been battle-tested way more in production, especially by the likes of Google. But from what I understand, Docker is really pushing swarm. I'd be curious to hear if others even considered Swarm before choosing K8s..

[lobster_johnson]

There's not really any comparison. Docker is clearly beefing up Docker/Swarm to be more like Kubernetes, but in its current state, Swarm is just a glorified Docker Compose.

For example, it does not handle services (K8s can automatically provision a load balancer against all your containers), there's no volume handling, no centralized logging, no label-based targeting, it has very limited scheduling (K8s uses cAdvisor to help scheduling, can automatically ensure that pods are spread out across multiple AZs, etc.), etc.

It'll be interesting to see what happens as Docker starts pushing into Kubernetes' space. Given the multiple points of overlap/contention between K8s and Docker (you have to disable Docker's built-in networking and iptables management; Kubelet has to continually monitor Docker for orphaned containers and volumes and so on; etc.) I wouldn't be surprised if Google one day decides to eliminate the Docker daemon as a dependency entirely, by writing a bare-bones container engine into Kubelet.

[nakagi]

Really? I also think Docker Swarm Mode is still behind of K8S, but as far as I read the doc, they support - load balancing between container - volume handling https://docs.docker.com/engine/tutorials/dockervolumes/ - label-based constraint

I know some features are not so sophisticated compared with K8S and there is no AZ awareness, but Swarm may try to catch up with it.

[lobster_johnson]

I recommend looking into the Kubernetes design to understand how different its design is.

A good example is volume management. With Kubernetes, you can tell a pod to use an AWS EBS volume; when the pod needs the volume, Kubernetes will automagically mount it, and handle the statement management for you.

If you define what's called a persistent volume, your pod can declare that it needs, say, 1GB, and Kubernetes will automatically allocate 1GB from the volume; you can have lots of pods working off this shared volume, and Kubernetes will know which pods have "claimed" which parts of the volume.

Another good example is config and secrets. In Kubernetes, you declaratively create configuration objects ("configmaps") and secrets. If a pod needs, say, access to an external API, you can store the keys in a secret and declaratively give the pod access to the secret, which will be mounted into a folder (or, alternatively, assigned to an environment variable, though that's not as secure).

Yet another example is service management. You can tag a service (which is another type declaration that says "port X on some unique cluster IP should be routed to every pod tagged with these labels") as load-balanced, and if you're running in a cloud environment (AWS, GCE, etc.), K8s can automatically create an external load balancer for you that exposes the service publicly.

Kubernetes is best described as a sophisticated state machine that takes declarative objects ("manifests") that describe your world — i.e. which containers should be running, which services should be exposed, etc. — and then attempts to continuously reconcile reality with that declaration, managing all sorts of state in the process.

Perhaps most important is the ability to abstract resources from pods. A pod just declares the image to run and the resources — volumes, configs, secrets, CPU/memory constraints, etc. — to make available to it. K8s's state machinery takes care of the rest.

As far as I know, Docker Swarm has none of this, and you'd have to build these things (e.g. REXRay for volumes) on top of Swarm yourself.

[nakagi]

Hmm, I just want to clarify you're talking about Docker Swarm of Docker v1.11- https://docs.docker.com/swarm/ or a new built-in Docker Swarm Mode from 1.12+. https://docs.docker.com/engine/swarm/

The latter obviously borrows a lot of design and concept from k8s, so I thought the design is not so different as previously they were. It just doesn't have some(or a lot, so far) cool features that k8s already provides (it's still in a RC stage)

[lobster_johnson]

Ah, I don't know the new Swarm Mode at all. A cursory look does make it seem like it's very much copying K8s.

[smarterclayton]

Not really Google driving this, but there is active work on integrating the OCI runtime (the "standard" evolution of libcontainer from Docker, used in Docker 1.12) as a container runtime to Kube. The desire is to reduce some of the overlap between container daemons on the nodes, but also support a wider array of container runtime setups (being able to run VMs via hyper, rkt containers, OCI containers, Docker containers, etc). Each of those mentioned technologies is being sponsored by different parts of the Kubernetes community, but the goal is to have more power and flexibility at runtime. Docker will continue to be a primary part of the story.