[binoct]

> You ended up with lots of magic an internal domain knowledge about how to do things, and often had to get clever to get the result you wanted.

How does Kubernetes not also result in the same? I'm not questioning plenty of improvements and the somewhat different domain it brings to the table overall but genuinely curious. From what I've seen any tool designed to integrate and manage complex, disparate systems is going to end up with lots special cases, domain knowledge, and require digging into implementation details on a regular basis for anything other than absolute common-case uses.

[gouggoug]

> How does Kubernetes not also result in the same?

Here's the difference between kubernetes and ansible (and alikes) (and I'm over-simplifying a bit of course, but the idea is there)

* With kubernetes, you write down the state you want your infrastructure to be in, and k8s figures out how to get there

* With ansible, you write down how to get to the state you want and ansible runs those steps, and hopefully they end up in the same state. The issue is that if you run the same set of steps on a Debian linux distribution, and on a Ubuntu distribution, well, they might end up in a similar state, but they actually aren't; for starters: one is debian, the other is unbuntu.

[dale_glass]

Ansible does try to do the 'write down the state you want your infrastructure to be in' thing. Eg,

    - name: "Ensure sshd is started and enabled on boot"
      systemd:
          state: started
          enabled: yes
          name: sshd

You're not saying 'enable the sshd service', but 'the desired state for sshd is started and enabled'.

But this is not perfect, because it's easy to eventually end up with imperative commands, like "run this script", or "Copy this file".

[kspacewalk2]

That does not seem avoidable. I have never used kubernetes, but this seems highly suspect:

>With kubernetes, you write down the state you want your infrastructure to be in, and k8s figures out how to get there

My guess is that this devolves to "run this script" or "copy this file" in practice as well. Or it just becomes tautological, i.e. "here's the state I want the system to be in (and that means run this script and then copy this file over here, and don't forget to symlink it over there)". Am I wrong?

[JeremyNT]

Kubernetes itself tries Very Hard to completely abstract those implementation details. Like, obsessively hard. You (assuming "you" are somebody who wants to run something in k8s) should never need to know or care what it's doing "under the hood" to run your pod in the way you say.

What's happened though is that Kubernetes expects for you to hand it an artifact that encapsulates all of the messy steps required before it can "run the thing." In practice that means the "run the script and symlink stuff" kind of glue is still present, it's just moved into Dockerfiles (which are, more or less, just fancy shell scripts themselves), build scripts, CI/CD pipelines, and startup scripts.

The big shift is more that the responsibility for managing this junk is pushed, organizationally, onto whoever knows how to operate the actual application (because they'll be creating the Dockerfile). Usually that's a developer.

So it's not wrong to say:

> With kubernetes, you write down the state you want your infrastructure to be in, and k8s figures out how to get there

But it's not really a complete explanation, because it assumes you've already gotten to the point that you've bundled an artifact up that will "do the right thing" when Kubernetes provides the fundamental pieces of infrastructure you define in your template.

[scrollaway]

Yeah you're (mostly) incorrect. I recommend looking into terraform and how it handles state, that will give you a better idea of how it's possible to do infrastructure in a "this is what i want, figure out how to get there" way.

Terraform isn't the best implementation of it; k8s has full control over the infra so it's even more powerful in that regard.

[pat2man]

Kubernetes doesn't attempt to be an API that can solve any orchestration issue, it is an abstraction on the underlying architecture. It's a lot easier to be backwards compatible here (and if you are not, create an operator that is).

Now whether its the right abstraction is a matter of debate. But ultimately like VMs or containers, its so far proven to be a valuable tool, at least for medium sized companies.

[saargrin]

and then you use something like helm to orchestrate and end up being in the same place where you left ansible:

lots of environment specific magic and flow-breaking with every upgrade of anything

same with cloud envs and terraform

[pm90]

One of the best things about K8s is its pretty stable API. We use GKE and the upgrades happen without issues 99% of the time.

The other benefit that k8s got for "free" was with containers: developers could stuff in whatever environment, software and configs they wanted inside the container. This removed most of the "my app broke because some shared lib didn't install properly" sorts of issues. This separation b/w software configuration and application configuration is extremely powerful.

[bostonsre]

Yea, I think with both ansible (and stuff like puppet) and kubernetes they are both powerful tools that are pretty leaky abstractions. They are great for sharing and implementing recipes, but that abstraction layer makes it a little more difficult to become a master chef and understand/control exactly what's happening. They are awesome to have in your toolbox, but I still find it incredibly useful to have a tool like python in there as well.

[colechristensen]

I’m not saying kube really fixes all of these problems or is a better solution, but it exists as a response to a crappy solution. in other words one problematic solution replaced another problematic solution. Stay in the field long enough and you see a pendulum where one kind of solution with problems is replaced by a different kind of solution with different problems... and then it swings back and a new general goes back to a new iteration of the first kind of solution and gets to rediscover the old problems again, back and forth over and over.

[waheoo]

Who said it doesn't? Look at its schema it's horrific.

The reason it's popular is that it's backed by Google and solves problems Docker the company couldn't.

That doesn't mean k8s is by any means not suffering the same problems of being a dsl that should be something else.

It also doesn't mean k8s won't suffer the same weirdness between platforms (AWS vs GCE vs GKE).