[lobster_johnson]

The problem is that it's not declarative. You can't tweak the config and run it again to converge.

Kubeup is designed to run once, unlike systems such as Puppet and Terraform that declaratively set up the world to fit your specification.

Kubeup also does a lot of mysterious stuff. By using it, you don't have a clear idea of which pieces have been set up and how they slot into each other. It is, in short, opaque and magical.

For comparison, I set up Kubernetes with Salt on AWS. It was, by all means, "the hard way", and took me a few days to get running and a couple of weeks to run completely correctly (a lot of stuff, like kubeconfig and TLS behaviour, is still undocumented), but as a byproduct I now have the entire setup in a reproducible, self-documenting, version-controlled config.

[DigitalJack]

Have you by chance open sourced your setup? I started going down this route with terraform, but ended up stopping and just using the kube-up script due to time constraints.

However, now that I have a cluster up and running, I can take the time to build a parallel cluster with more understanding, and migrate the services to it.

I found a terraform example, but it declares itself out of date, and looked more complicated than I thought it should be... that was just a gut feel though.

I have not used Salt, but I always like to learn new things, especially if they make my life easier.

I'd be very interested in checkout out your setup, and any lessons learned you have to share.

Thanks in advance

[marcoceppi]

We've opensourced our setup: http://www.jorgecastro.org/2016/07/29/ubuntu-kubernetes-v1-d...

[DigitalJack]

This project looks very interesting, and once I'm more comfortable at a lower level of abstraction, I may use this.

[lobster_johnson]

I haven't, but I would be happy to.

I just need to generalize it a little bit. Email me and I will send you a link once it's done, sometime next week (I'm on vacation).

[nathan_f77]

Hi, I would just like to second the request for an open-source, production-ready implementation. They seem to be rarely shared, which is a big shame.

I've been playing with https://github.com/kz8s/tack lately, but your implemention sounds like even more comprehensive.

[manojlds]

I used this as reference - https://github.com/samsung-cnct/kraken

Terraform + Ansible for K8S on CoreOS

[DigitalJack]

Thanks for the tip. I will take a closer look at this.

[olalonde]

Oh yeah I agree, that's annoying. I considered using one of the few projects that attempt to solve this problem but decided to stick with `kube-up.sh` because, as a beginner, I'd have a hard time telling which pieces belong to Kubernetes vs the third party tool. I also don't have time to become a Kubernetes expert because I'm crazy busy developing. Hopefully, Kubernetes will eventually obsolete `kube-up.sh` for something better based on Salt/CloudFormation/Terraform/etc.

[manojlds]

Watch this tool called kops - https://github.com/kubernetes/kops

Official upcoming replacement for kube-up

[lobster_johnson]

Kube-up actually uses Salt for some of the setup. But it's a big mess, since it needs to support a lot of platforms, Linuxes and cloud providers.

I don't know what the future of Kubernetes setup is, exactly, but right now it's quite safe to settle on Salt, Puppet, Ansible or Terraform. I haven't used Terraform, so I don't know how suitable it is to OS-level setup (things that the aforementioned tools are good at), as opposed to orchestration.

[elsonrodriguez]

When you click the GKE(hosted Kubernetes) button on Google Cloud Platform, it's those very Salt configs that set up your nodes (and once upon a time they setup your control plane too).

Basically what's arguably the best publicly available Kubernetes setup in the world is hiding in that Salt codebase, and EVERY would-be Kubernetes admin should look at it before venturing on their own.

[lobster_johnson]

I used it as inspiration for my setup. But you also need a bunch of other stuff, like the CA setup and Kubelet cert generation, which are buried in the whole kube-up structure.

[olalonde]

Yeah, agreed. CoreOS + cloud-init could remove necessity to do OS-level setup.

[elsonrodriguez]

There's always something to do at the OS-level, which is why cloud-init configurations tend to spiral out of control with in-line scripts, configs, and binary downloads.

There's no getting away from configuration management and software installation at SOME level of your stack, and setting up a substrate for Kubernetes is no exception.

[roman_sf]

Nah, who needs all these version-controlled configs?

Just look how google runs stuff! https://cloud.google.com/compute/docs/tutorials/setup-joomla