[willejs]

This all makes me laugh, and cry at the same time. It makes me laugh because everyone wants to run k8s for no real reason, they havent got scale, traffic or many woes. Please just run some vms, CM, unattended upgrades, capistrano and packer. Mostly the loose reasoning is 'simplicity', and its the new shiny. This is perceived by people thinking that deployment, service discovery, config etc. is provided for free in kubernetes, and one boot script will solve all. On top, everyone thinks its trivial to operate this, maintain it, and no one understands what 'production ready is'. I almost think people think it replaces ops, but it does the opposite.

It makes me cry because, running k8s is hard, ops is hard, and so is telling people they might be wrong. K8s consists of half a dozen components, they have dozens of config flags, and much functionality is buggy, in beta, or flux. To top this off k8s is based on etcd. Etcd is barely production ready by their own admissions (remember /production.md in github?) but if you have run it you will understand the bugs, and vague docs coupled with reading the source constantly when problems arise. K8s consists of many components, kubelet, proxy, controller, scheduler and more. These you have to install and configure, and many scripts do this badly in a one size fits all approach, and many CM methods do this barely in an ok manner currently. I cry, because of overlay networking too, its a nightmare, and the alternative cloud permissions are scary.

[siegecraft]

How do you reconcile "ops is hard" with "just run some vms, CM, etc."? Is it because ops only becomes hard when you force yourself to use k8 when it's not really needed?

Also do people really think k8 is a drop-in/trivial solution? I just got done evaluating it and the overriding sentiment seemed to be "it's super flexible but super complex and badly documented and you'd better hope you're using the happiest of happy paths."

[heurist]

My anecdotal experience: I only started working with Docker and distributed systems in January and I've been through a couple iterations of provisioning, deployment, and orchestration since then - started with docker-machine and docker-compose, added in swarm, private registries, and a set of bash scripts, now moving to kubernetes - and I am finding that kubernetes handles many tricky components out of the box with very little effort. I wouldn't have understood how to use it four months ago but now that I've implemented much of it myself I understand the underlying architecture and design goals and find it to be a better solution than the tools I've put together. It took me a few days to see if it would do everything I needed and understand how to configure it, which I suppose could be seen as complexity, but it took me much longer to understand how to run docker in production in the first place since I had to learn and build it all without prior knowledge. Kubernetes has been a cakewalk in comparison. If I ever need to set up a cluster from scratch again I'll be using k8s.

[TheIronYuppie]

Disclosure: I work at Google on Kubernetes.

We're always looking to improve on both complexity and documentation - but, as Kelsey pointed out in his tutorial, it's definitely got a few steps. The biggest question is would you really want any fewer? That is to say, you could use a hosted solution (like Google Container Engine) or set up with a one-line command, but most people who want to run at production scale definitely want individual steps so they can customize.

[TheIronYuppie]

Hi - I work at Google on Kubernetes.

The biggest reason people use Kubernetes is because it is the best solution to solve the need "how do I bring my containers to production." You're absolutely right, you can solve this in many other ways, but, no matter what, you're going to have to address the problem of scaling, updating, monitoring, logging, securing, maintaining persistence, implementing 12-factor applications and many many other things before you do - Kubernetes solves all this for you.

Just a quick correction - Kubernetes is not based on etcd, nor do you have to set up half a dozen components. There are three components to Kubernetes - an API server, a scheduler and a controller manager - there are flags, true, but you can easily use a configuration file (just as you would with any other production ready server).

If you saw anything that's badly documented or buggy, please let me know!

[jdubs]

k8s is still way easier to run than mesos & marathon with all the accompanying service that make it usable.

[leetrout]

And Nomad is an order of magnitude easier than k8s. Might not be as full featured but for basic use cases of give me X resources to run Y it's really great. I was shocked at how easy it is to setup and run. The folks at Hashicorp are doing some great things.

[SEJeff]

Agreed (as someone who has spent the last 1.5 years working on/with Mesos and will soon be looking to migrate to openshift/k8s)

[XorNot]

"unattended upgrades".

Why? With VMs there is no reason, ever, to be modifying the OS with the VM still running, rather then cutting over to a new, tested instance.

[willejs]

Sorry, I was referring to security updates through unattended updates. Its pretty hard to do if your machine is storing state or is running a lot of traffic. Whilst churning your entire estate should be possible, it is not always.