[throwaway894345]

> So rather than being like systemd, which adds some complexity and also takes some away, Kubernetes only adds.

Here are some bits of complexity that managed Kubernetes takes away:

* SSH configuration

* Key management

* Certificate management (via cert-manager)

* DNS management (via external-dns)

* Auto-scaling

* Process management

* Logging

* Host monitoring

* Infra as code

* Instance profiles

* Reverse proxy

* TLS

* HTTP -> HTTPS redirection

So maybe your point was "the VMs still exist" which is true, but I generally don't care because the work required of me goes away. Alternatively, you have to have most/all of these things anyway, so if you're not using Kubernetes you're cobbling together solutions for these things which has the following implications:

1. You will not be able to find candidates who know your bespoke solution, whereas you can find people who know Kubernetes.

2. Training people on your bespoke solution will be harder. You will have to write a lot more documentation whereas there is an abundance of high quality documentation and training material available for Kubernetes.

3. When something inevitably breaks with your bespoke solution, you're unlikely to get much help Googling around, whereas it's very likely that you'll find what you need to diagnose / fix / work around your Kubernetes problem.

4. Kubernetes improves at a rapid pace, and you can get those improvements for nearly free. To improve your bespoke solution, you have to take the time to do it all yourself.

5. You're probably not going to have the financial backing to build your bespoke solution to the same quality caliber that the Kubernetes folks are able to devote (yes, Kubernetes has its problems, but unless you're at a FAANG then your homegrown solution is almost certainly going to be poorer quality if only because management won't give you the resources you need to build it properly).

[dolni]

Respectfully, I think you have a lot of ignorance about what a typical cloud provider offers. Let's go through these each step-by-step.

> SSH configuration

Do you mean the configuration for sshd? What special requirements would have that Kubernetes would help fulfill?

> Key management

Assuming you mean SSH authorized keys since you left this unspecified. AWS does this with EC2 instance connect.

> Certificate management (via cert-manager)

AWS has ACM.

> DNS management (via external-dns)

This is not even a problem if you use AWS cloud primatives. You point Route 53 at a load balancer, which automatically discovers instances from a target group.

> Auto-scaling

AWS already does this via autoscaling.

> Process management

systemd and/or docker do this for you.

> Logging

AWS can send instance logs to CloudWatch. See https://docs.aws.amazon.com/systems-manager/latest/userguide....

> Host monitoring

In what sense? Amazon target groups can monitor the health of a service and automatically replace instances that report unhealthy, time out, or otherwise.

> Infra as code

I mean, you have to have a description somewhere of your pods. It's still "infra as code", just in the form prescribed by Kubernetes.

> Instance profiles

Instance profiles are replaced by secrets, which I'm not sure is better, just different. In either case, if you're following best practices, you need to configure security policies and apply them appropriately.

> Reverse proxy

AWS load balancers and target groups do this for you.

> HTTPS

AWS load balancers, CloudFront, do this for you. ACM issues the certificates.

I won't address the remainder of your post because it seems contingent on the incorrect assumption that all of these are "bespoke solutions" that just have to be completely reinvented if you choose not to use Kubernetes.

[throwaway894345]

> I won't address the remainder of your post because it seems contingent on the incorrect assumption that all of these are "bespoke solutions" that just have to be completely reinvented if you choose not to use Kubernetes.

You fundamentally misunderstood my post. I wasn't arguing that you had to reinvent these components. The "bespoke solution" is the configuration and assembly of these components ("cloud provider primitives" if you like) into a system that suitably replaces Kubernetes for a given organization. Of course you can build your own bespoke alternative--that was the prior state of the world before Kubernetes debuted.

[dolni]

That's not really any different for Kubernetes.

You still need to figure out where your persistent storage is.

You still have to send logs somewhere for aggregation.

You have the added difficulty of figuring out cost tracking in Kubernetes since there is not a clear delineation between cloud resources.

You have to configure an ingress controller.

You want SSL? Gotta set that up, too.

You have to figure out how pods are assigned to nodes in your cluster, if separation of services is at all a concern (either for security or performance reasons).

Kubernetes is no better with the creation of "bespoke solutions" than using what your cloud provider offers.

Compare this tutorial for configuring SSL for Kubernetes services to an equivalent for configuring SSL on an AWS load balancer. Is Kubernetes really adding value here?

https://blog.karmacomputing.co.uk/kubernetes-cluster-with-ss... https://aws.amazon.com/premiumsupport/knowledge-center/assoc...

[parasubvert]

Kubernetes is far better for each of the above tasks because it is a consistent approach and set of abstractions rather than looking through the arbitrary "everything store" of the cloud providers. I really don't have any interest in relying on 15 different options from cloud providers, I want to get going with a set of extensible, composable abstractions and control logic. Software should not be tied to the hardware I rent or the marketing whims of said entity.

Yes, there is choice and variety among Kubernetes extensions, but they all have fundamental operational assumptions that are aligned because they sit inside the Kubernetes control and API model. It is a golden era to have such a rich set of open and elegant building blocks for modern distributed systems platform design and operations.

[throwaway894345]

Well, first of all, note how much shorter your list is than the original. So vanilla Kubernetes is already taking care of lots of things for us (SSH configuration, process management, log exfiltration, etc). Moreover, we're not talking about vanilla Kubernetes, but managed Kubernetes (I've been very clear and explicit about this) so most of your points are already handled.

> You still need to figure out where your persistent storage is.

Managed Kubernetes comes with persistent storage solutions out of the box. I don't know what you mean by "figure out where it is". On EKS it's EFS, on GKE it's FileStore, and of course you can use other off-the-shelf solutions if you prefer, but there are defaults that you don't have to laboriously set up.

> You still have to send logs somewhere for aggregation.

No, these too are automatically sent to CloudWatch or equivalent (maybe you have to explicitly say "use cloudwatch" in some configuration option when setting up the cluster, but still that's a lot different than writing ansible scripts to install and configure fluentd on each host).

> You have the added difficulty of figuring out cost tracking in Kubernetes since there is not a clear delineation between cloud resources.

This isn't true at all. Your cloud provider still rolls up costs by type of resource, and just like with VMs you still have to tag things in order to roll costs up by business unit.

> You have to configure an ingress controller.

Nope, this also comes out of the box with your cloud provider. It hooks into the cloud provider's layer 7 load balancer offering. It's also trivial to install other load balancer controllers.

> You want SSL? Gotta set that up, too. ... Compare this tutorial for configuring SSL for Kubernetes services to an equivalent for configuring SSL on an AWS load balancer. Is Kubernetes really adding value here?

If you use cert-manager and external-dns, then you'll have DNS and SSL configured for every service you ever create on your cluster. By contrast, on AWS you'll need to manually associate DNS records and certificates with each of your load balancers. Configuring LetsEncrypt for your ACM certs is also quite a lot more complicated than for cert-manager.

> Kubernetes is no better with the creation of "bespoke solutions" than using what your cloud provider offers.

I hope by this point it's pretty clear that you're mistaken. Even if SSL/TLS is no easier with Kubernetes than with VMs/other cloud primitives, we've already addressed a long list of things you don't need to contend with if you use managed Kubernetes versus cobbling together your own system based on lower level cloud primitives. And Kubernetes is also standardized, so you can rely on lots of high quality documentation, training material, industry experience, FAQ resources (e.g., stack overflow), etc which you would have to roll yourself for your bespoke solution.