[chrissnell]

It's really not that difficult to network containers. We're using flannel [1] on CoreOS. We're using flannel's VXLAN backend to encapsulate container traffic. We're Kubernetes users so every kube pod [2] gets it's own subnet and flannel handles the routing between those subnets, across all CoreOS servers in the cluster.

I was skeptical when we first deployed it but we've found it to be dependable and fast. We're running it in production on six CoreOS servers and 400-500 containers.

We did evaluate Project Calico initially but discovered some performance tests that tipped the scales in favor of flannel. [3] I don't know if Calico has improved since then, however. This was about a year ago.

[1] https://github.com/coreos/flannel

[2] A Kubernetes pod is one or more related containers running on a single server

[3] http://www.slideshare.net/ArjanSchaaf/docker-network-perform...

[crb]

Even better, Flannel and Calico have merged. See:

https://www.projectcalico.org/canal-tigera/

https://coreos.com/blog/coreos-intel-calico-packet-extend-gi...

https://github.com/tigera/canal

[moondev]

Is flannel used in Kubernetes for networking by default? Or is it something that needs to enabled and configured separately?

[amouat]

Kubernetes has a requirement that containers (more accurately "pods") can connect via a "flat networking space". How this is achieved varies between deployments, flannel, calico and weave are all common approaches. Kelsey Hightower's "Kubernetes the Hard Way" simply configured it at the router level: https://github.com/kelseyhightower/kubernetes-the-hard-way/b...

[moondev]

That makes sense. Thanks for the link as well, i've been looking for something exactly like it. Looks like a great resource!

[lobster_johnson]

Kubernetes doesn't have a "default" as such. It requires something external to manage the subnet, and needs to be configured to use it.

However, if you run it on AWS, it can automatically configure a bridge (cbr0) and configure up the VPC routing table for you.

GCE (Google's managed Kubernetes on Google Cloud) also handles this automatically.

There's also experimental support for Flannel built into K8s, which can be enabled with a flag. Not sure if it's worth using.

[bboreham]

Nitpick: Google's managed Kubernetes is called GKE.

However the OSS Kubernetes has code to configure routes on GCE same as it does for AWS.