[SOLAR_FIELDS]

It's yet another argument, like many arguments against Kubernetes, that essentially boils down to "Kubernetes is complicated!"

No, deployment of a distributed system itself is complicated, regardless of the platform you deploy it on. Kubernetes is only "complicated" because it can do all of the things you need to do to deploy software, in a standard way. You can simplify by not using Kube, but then you have to hand roll all of the capabilities that Kube just gives you for free. If you don't need a good hunk of those capabilities, you probably don't need Kube.

[bigstrat2003]

I think the reason why people (correctly) point out that Kubernetes is complicated is because most people do not need a distributed system. People reach for k8s because it's trendy, but in truth a lot of users would be better off with a VM that gets configured with Chef/etc and just runs your software the old fashioned way.

[zomiaen]

K8s starts to make sense when you want to provide a common platform for a multitude of application developers to work on. Once you can understand it was born from Google's Borg and what problems they were trying to solve with both, the complexity behind it makes a lot more sense.

[vbezhenar]

Most people actually do need a distributed system.

They want their system to be reliable to hardware failures. So when the server inevitably goes down some day, they want their website to continue to work. Very few people wants their website to go down.

They want their system to scale. So when the sudden rise of popularity hits the load balancer, they want their website to continue to work.

In the past, the price to run a distributed system was too high, so most people accepted the downsides of running a simple system.

Nowadays the price to run a distributed system is so low, that it makes little sense to avoid it anymore, for almost any website, if you can afford more than $50/month.

[aljgz]

Very well put. I add to that: with an E2E solution, you need to learn things before you deploy your system (still not that you do everything properly), but without that, it's possible to deploy the system, then learn things can go wrong when they actually happen. Now if you ask someone who has a half baked distributed system, they still don't know all the failure modes of their system. I've seen this in mission-critical systems.

But in a company that had properly reliable infrastructure, any system that moved to the new infra based on K8s needed much less maintenance, had much more standardized DevOps (which allowed people from other teams to chime in when needed), and had much fewer mistakes. There was no disagreement that K8s stramlined everything.

[clvx]

I really want NixOS to succeed in becoming easy to configure, operate and onboard. I would like to avoid any other configuration management tool in my life again. In this way, you can have simple vm's and k8s for distributed use cases. Both that can be declared in code.

[knowsuchagency]

well said

[danpalmer]

Exactly, there are a lot of comparisons that aren't apples to apples. If you're comparing kubernetes to a fixed size pool of resources running a fixed set of applications each with their own fixed resources, who cares? That's not how most deployments today.

One could make the argument that deployments today that necessitate K8s are too complex, I think there's a more convincing argument there, but my previous company was extremely risk averse in architecture (no resumé driven development) and eventually moved on to K8s, and systems at my current company often end up being way simpler than anyone would expect, but at scale, the coordination without a K8s equivalent would be too much work.

[bigfatkitten]

Kubernetes is often chosen because the requirement for resume and promo pack padding has taken precedence over engineering requirements.

Organisations with needs big and complex enough for something like Kubernetes are big enough to have dedicated people looking after it.

[nostrebored]

When choosing distributed systems platforms to work with, k8s vs. rolling your own orchestration isn’t the decision anyone is making. It’s k8s vs cloud vendors that want your money in exchange for the headaches.

[SOLAR_FIELDS]

Honestly running your own control plane is not that much harder than using something like EKS or GKE. The real complexity that the grandparent was talking about is all the tweaking and configuration you have to do outside of the control plane. Eg the infrastructure and deployments you’re building on top of Kubernetes and all of the associated configuration around that. In other words, whether you use EKS or hand roll your own kube you still have to solve node auto scaling. Load balancing. Metrics/observability. DNS and networking. Ingress. Etc etc etc.

[nostrebored]

Right, but I didn’t mention EKS. 99% of companies adopting k8s shouldn’t and have no business case for the technology.