|
The last time I used ECS for a production deploy, you could group containers together (just as you can on compose). However, there were no easy way to do service discovery. This made wiring containers together difficult. If I wanted one container to talk to another, I had to group them and deploy them as one unit. That meant I could not horizontally scale one container more than the other. I can scale the whole group, but there is a lot of wasted resources at that point. Kubernetes pods group containers together under a single IP address. Containers from one pod (one IP address) can talk to any other pod. Docker did not even have this functionality until 1.12, and that is too little, too late. (And I am not sure this is something ECS supports right now). Combined with label selectors, long-running services (which binds a DNS name to the set selected by the label selectors), I can horizontally-scale pods and still maintain service discoverability. Using DNS makes service discovery stupid-easy. This means I can scale Kubernetes pods independently from each other. Another consequence of using Service objects to select a set based on label selectors is routing can now be dynamic. Pods that need to talk to another pod goes through the service. I can then scale the dependency up and down, and it doesn't really affect the pod that requires that service. I can do rolling upgrades to the dependency, and it works because Service abstracts that through label selectors. There are still some warts related to this setup. Stateful sets still needs a lot of work. I've also found that many applications caches IP addresses (redis sentinel being a notorious example). To work well with Kubernetes, it's better to always query DNS when making a connection. Ruby drivers for Mongodb and Redis, for example, will cache DNS lookup, making failover fragile (if you are running Mongodb and Redis inside Kubernetes; if you're not, you won't have this problem). I was choosing between Kubernetes and Mesos after ECS, but had not looked into either deeply. It was random chance that took me to Kubernetes instead of Mesos. Kubernetes solved many of the pain points of Docker Compose and ECS. |