[cortesoft]

Do people try to push it that strongly for small teams? Lots of us work on bigger teams and enjoy more of the benefits.

However, I also still use Kubernetes for my personal projects, because I really appreciate the level of abstraction it supplies. Everyone always points out that you can do all the things k8s does in other ways, but what I like about it defines a common way to do everything. I don't care that there are 50 ways to do it, I just like having one way.

What this allows is for tools to seamlessly work together. It is trivial to have all sorts of cool functionality with minimal configuration.

[datadeft]

> because I really appreciate the level of abstraction it supplies

which are?

I am seriously asking. I use docker-compose of some of the things I do but it never occured to me during my 20 years in systems engineering that k8s offers any kind of great abstraction. For small systems it is easy to use docker (for example running a database for testing). For larger projects there are so many aternatives to k8s that are better, including the major cloud vendor offerings that I have really a hard time justifying even to consider k8s. After years of carnage that they left, seeing failures after failures, even customers reaching out to me in panic to help them because there are timeouts or other issues that nobody can resolve after selling them the idea that k8s has "great level of abstraction" and putting it to production.

> I don't care that there are 50 ways to do it, I just like having one way.

Seeing everything as a nail...

[cortesoft]

>> because I really appreciate the level of abstraction it supplies

> which are?

When I am creating a new service/application, I just need to define in my resource what I need... listening ports, persistent storage, CPU, memory, ingress, etc... then I am free to change how those are provided without having to change the app. If a new, better, storage provider comes along, I can switch it out without changing anything on my app.

At my work, we have on premise clusters as well as cloud clusters, and I can move my workloads between them seamlessly. In the cloud, we use EBS backed volumes, but my app doesn't need to care. On the on-prem clusters, we use longhorn, but again my app doesn't care. In AWS, we use the ELB as our ingress, but my app doesn't care... on prem, I use metallb, but my app doesn't care.

I just specify that I need a cert and a URL, and each cluster is set up to update DNS and get me a cert. I don't have to worry about DNS or certs expiring. When I deploy my app to a different cluster, that all gets updated automatically.

I also get monitoring for free. Prometheus knows how to discover my services and gather metrics, no matter where I deploy. For log processing, when a new tool comes out, I can plug it in with a few lines of configuration.

The kubernetes resource model provides a standard way to define my stuff. Other services know how to read that resource model and interact with it. If I need something different, I can create my own CRD and controller.

I am able to run a database using a cluster controller with my on prem cluster without having to manage individual nodes. Anyone who has run a database cluster manually knows hardware maintenance or failure is a whole thing... with controllers and k8s nodes, I just need to use node drain and my controller will know how to move the cluster members to different nodes. I can update and upgrade the hardware without having to do anything special. Hardware patching is way easier.

The k8s model forces you to specify how your service should handle node failure, and nodes coming in or out are built into the model from the beginning. It forces you to think about horizontal scaling, failover, and maintenance from the beginning, and gives a standard way for it to work. When you do a node drain, every single app deployed to the cluster knows what to do, and the maintainer doesn't have to think about it.

>> I don't care that there are 50 ways to do it, I just like having one way.

> Seeing everything as a nail...

I don't think that is a fair comparison, because you can create CRDs if your model doesn't fit any existing resource. However, even when you create a CRD, it is still a standard resource that hooks into all of the k8s lifecycle management, and you become part of that ecosystem.

[datadeft]

> listening ports, persistent storage, CPU, memory, ingress

These exists without k8s. I do not need a compex abstraction hiding the ways I need to talk to persistent storage. If fact, I believe it is impossible to create such abstraction without very serious compromises.

> In AWS, we use the ELB as our ingress, but my app doesn't care

Your app does not care without k8s. Running python -m http.server does not even know what ELB is. I get it though. You are using k8s as IaC.

[cortesoft]

> These exists without k8s.

That was exactly my earlier point... of course you can do everything in k8s in other ways, but in the end you have to pick ONE way your company/team is going to do it... why not pick a well defined way, that new hires can already know, that has a ton of tooling available, and works together cohesively?

Yes, I can build each part myself, but why?

> Your app does not care without k8s. Running python -m http.server does not even know what ELB is. I get it though. You are using k8s as IaC.

Sure, but I still need a way to deploy my app, and to move it to a different location when I do hardware maintenance, and a way to get a DNS address that routes to my app.

At my shop, using k8s, I can deploy a brand new service, with a cert, a url, and a place to run it, in a few minutes. I don't have to talk to anyone, I don't have to use any other tools or have to click on any buttons, i just helm install or kubectl apply and my service is running. I don't have to ask the datacenter ops people to find me a server, or get budget for a new AWS instance. I can deploy to an existing cluster and use a small bit of the infrastructure. I don't have to scale my individual service, I can scale the whole cluster for all services.

It is just so much easier to be a developer in this world.

[dilyevsky]

> I do not need a compex abstraction hiding the ways I need to talk to persistent storage. If fact, I believe it is impossible to create such abstraction without very serious compromises.

That's pretty interesting take considering EBS is itself a block device abstraction over network attached storage and pretty complex at that too with a huge price premium

[cortesoft]

I mean, the file system istelf is an abstraction.

[dilyevsky]

yet nobody arguing it's useless when your admin lost all your data bc he used btrfs with raid5 or zfs on linux, incredible!

[nostrebored]

Yes. Whenever I look at a company with less than 20 people with EKS in their stack, I don't go any further. It is such a colossal waste of velocity for a small business or early startup.

As someone who is very pro cloud -- one of my worst experiences working at a cloud provider was a push from on high to sell our customers on a 'cloud modernization initative' that centered on managed kubernetes. At the time, most of my customers were struggling with creating a stateless app, much less horizontal scaling and managing an enterprise-grade compute abstraction layer.

I think K8S is a great tool with a dedicated team and a platform built around it to meet the way that your company ships infrastructure. But what I've just mentioned only makes sense fiscally in the high X00's count or more of engineers.

[politelemon]

> Do people try to push it that strongly for small teams?

Yes. You have to understand that a lot of people without the benefit of experience will often base their technology choices on blog posts. K8S has a lot of mindshare and blog attention, so it gets seen as the only way to run a container in a production environment, while all the important aspects of it are ignored.

[cortesoft]

I get that, but I just get frustrated in the same way I get frustrated with all the "you don't need it" responses to any topic... what about all of us that DO work for bigger companies and DO need to use this stuff? Where can we gather to talk about it without being constantly told we don't need the features?

[ehutch79]

They don’t read those blogs. And if they do, the decision makers have enough experience to know that “your dog blog doesn’t need k8s” doesn’t apply to their 100000 mau app

[cortesoft]

I am literally one of the decision makers at a larger company, with more than 10000 servers in hundreds of data centers around the world.

Yes, I am experienced and smart enough to know the statements that don't apply to me. My frustration is that I want to discuss the best tools and techniques the industry is exploring, but every time I start to have those conversations, someone comments that I don't need it.

[ehutch79]

You're in the wrong spaces. I don't know where you should be to have those conversations, but I imagine it involves (social / interpersonal) networking. You need to be talking to people in the same role or at the same level as you.

Places like hacker news, or reddit, or twitter, are all full of random people, many of whom are just beginning their journey. Recommending multi node orchestration when they'd struggle to get nginx running on it's own, would be inappropriate. They don't need k8s. There's a significant danger of cargo culting here.

[justinclift]

> I also still use Kubernetes for my personal projects

Of course. You're going with what you know for the foundation, then building the thing you're interested in (personal projects) on top of that.

It's a good way of using your time efficiently. Nothing at all wrong with that.

[karolist]

This. It's the npm install 100 packages and do everything with JS vs Rails arguments all over again.