[datadeft]

> because I really appreciate the level of abstraction it supplies

which are?

I am seriously asking. I use docker-compose of some of the things I do but it never occured to me during my 20 years in systems engineering that k8s offers any kind of great abstraction. For small systems it is easy to use docker (for example running a database for testing). For larger projects there are so many aternatives to k8s that are better, including the major cloud vendor offerings that I have really a hard time justifying even to consider k8s. After years of carnage that they left, seeing failures after failures, even customers reaching out to me in panic to help them because there are timeouts or other issues that nobody can resolve after selling them the idea that k8s has "great level of abstraction" and putting it to production.

> I don't care that there are 50 ways to do it, I just like having one way.

Seeing everything as a nail...

[cortesoft]

>> because I really appreciate the level of abstraction it supplies

> which are?

When I am creating a new service/application, I just need to define in my resource what I need... listening ports, persistent storage, CPU, memory, ingress, etc... then I am free to change how those are provided without having to change the app. If a new, better, storage provider comes along, I can switch it out without changing anything on my app.

At my work, we have on premise clusters as well as cloud clusters, and I can move my workloads between them seamlessly. In the cloud, we use EBS backed volumes, but my app doesn't need to care. On the on-prem clusters, we use longhorn, but again my app doesn't care. In AWS, we use the ELB as our ingress, but my app doesn't care... on prem, I use metallb, but my app doesn't care.

I just specify that I need a cert and a URL, and each cluster is set up to update DNS and get me a cert. I don't have to worry about DNS or certs expiring. When I deploy my app to a different cluster, that all gets updated automatically.

I also get monitoring for free. Prometheus knows how to discover my services and gather metrics, no matter where I deploy. For log processing, when a new tool comes out, I can plug it in with a few lines of configuration.

The kubernetes resource model provides a standard way to define my stuff. Other services know how to read that resource model and interact with it. If I need something different, I can create my own CRD and controller.

I am able to run a database using a cluster controller with my on prem cluster without having to manage individual nodes. Anyone who has run a database cluster manually knows hardware maintenance or failure is a whole thing... with controllers and k8s nodes, I just need to use node drain and my controller will know how to move the cluster members to different nodes. I can update and upgrade the hardware without having to do anything special. Hardware patching is way easier.

The k8s model forces you to specify how your service should handle node failure, and nodes coming in or out are built into the model from the beginning. It forces you to think about horizontal scaling, failover, and maintenance from the beginning, and gives a standard way for it to work. When you do a node drain, every single app deployed to the cluster knows what to do, and the maintainer doesn't have to think about it.

>> I don't care that there are 50 ways to do it, I just like having one way.

> Seeing everything as a nail...

I don't think that is a fair comparison, because you can create CRDs if your model doesn't fit any existing resource. However, even when you create a CRD, it is still a standard resource that hooks into all of the k8s lifecycle management, and you become part of that ecosystem.

[datadeft]

> listening ports, persistent storage, CPU, memory, ingress

These exists without k8s. I do not need a compex abstraction hiding the ways I need to talk to persistent storage. If fact, I believe it is impossible to create such abstraction without very serious compromises.

> In AWS, we use the ELB as our ingress, but my app doesn't care

Your app does not care without k8s. Running python -m http.server does not even know what ELB is. I get it though. You are using k8s as IaC.

[cortesoft]

> These exists without k8s.

That was exactly my earlier point... of course you can do everything in k8s in other ways, but in the end you have to pick ONE way your company/team is going to do it... why not pick a well defined way, that new hires can already know, that has a ton of tooling available, and works together cohesively?

Yes, I can build each part myself, but why?

> Your app does not care without k8s. Running python -m http.server does not even know what ELB is. I get it though. You are using k8s as IaC.

Sure, but I still need a way to deploy my app, and to move it to a different location when I do hardware maintenance, and a way to get a DNS address that routes to my app.

At my shop, using k8s, I can deploy a brand new service, with a cert, a url, and a place to run it, in a few minutes. I don't have to talk to anyone, I don't have to use any other tools or have to click on any buttons, i just helm install or kubectl apply and my service is running. I don't have to ask the datacenter ops people to find me a server, or get budget for a new AWS instance. I can deploy to an existing cluster and use a small bit of the infrastructure. I don't have to scale my individual service, I can scale the whole cluster for all services.

It is just so much easier to be a developer in this world.

[dilyevsky]

> I do not need a compex abstraction hiding the ways I need to talk to persistent storage. If fact, I believe it is impossible to create such abstraction without very serious compromises.

That's pretty interesting take considering EBS is itself a block device abstraction over network attached storage and pretty complex at that too with a huge price premium

[cortesoft]

I mean, the file system istelf is an abstraction.

[dilyevsky]

yet nobody arguing it's useless when your admin lost all your data bc he used btrfs with raid5 or zfs on linux, incredible!