[Cwizard]

Could someone explain to me in what why Kubernetes is complex and what alternatives are simpler? I’ve worked on non-k8s systems before and in my experience they all hang together with custom bash/python code which although line-for-line is ‘simpler’ it makes it harder to onboard new people and is less robust (excluding very simple deployments)

K8s is very modular in my experience so if you don’t need something you can easily ignore it and not pay a complexity cost. Nomad does not seem much simpler to me (especially because you basically have to pair it with Consul and Vault)

I am genuinely curious.

[xyzzy123]

Mostly agree, I prefer k8s to learning the custom duct tape for every project.

Observation: a side effect of being extensible is that people deploy extensions.

There is some kind of law of complexity budgets, where if you make the simple things easy, people will tend to ratchet up complexity by adding more stuff until the system "just" fits in their heads again.

Bare k8s with a simple ingress path and workload is predictable and nice to admin.

Cluster with lots of extra bits (custom autoscalers, cert-manager, complex ci systems, serverless stuff, custom operators, service meshes) can have lots of "non-local" interactions and seems to lead to environments that are scary to upgrade.

[marginalia_nu]

You kinda need to still learn the custom duct tape though. Kubernetes is something you need to learn on top of all the details. It doesn't replace it. (It may however seem that way until you run into any actual problems with your software)

[thunky]

Yeah it's more like a network of duct tape webbed together within in a clear plastic box with a "kubectl" button on it along with a paper feeder that takes a stack of generated config files as input for each button press.

And this controls how the duct tape sticks everything together, so it's not like any two boxes are the same.

[kodah]

I guess it depends on what you're comparing Kubernetes to. If what's being compared is a large codebase with lots of loosely connected cloud formation, then Kubernetes can make your life easier. If what's being compared is a small application that can run on a collection of EC2's behind an ASG and easily deployed via Terraform then it's probably complex for that usecase.

It's all relative.

[mnomitch]

Nomad does not seem much simpler to me (especially because you basically have to pair it with Consul and Vault)

Hi, Nomad PM here - We've gotten this feedback a lot and have been taking steps to respond to it. We added simple service discovery in Nomad 1.3 and health checks and load balancing shortly after. So you shouldn't need Consul until you want a full service mesh. And then in Nomad 1.4, which just launched, we added Nomad Variables. These can be used for basic secrets & config management. It isn't a full on replacement for Vault, but it should give people the basic functionality to get off the ground.

So going forward we won't have a de factor dependency on these other tools, and hopefully we can live up to the promise of simplicity.

[nuker]

AWS ECS