[vbezhenar]

Kubernetes is weird and unique software. I feel for it to be more like underlying layer, rather than end solution yet I end up using it as a solution, because all layers just add complexity.

At the same time its popularity makes it obvious that competition basically will not exist in the foreseeable future.

I'm not sure that I qualified to judge it, but I feel that Kubernetes has great architecture.

I was astounded when I realized that Kubernetes is super flexible when it comes to its underlying data model and API. At some time I wanted to reimplement it as a data store and I still think that it might make sense. Think about PostgREST but much more mature and flexible. Like you submit CRD which is JSON schema and then you can insert your entities, query them, listen for changes and so on. And organize your application as a set of "controllers" reconciling those entities into some final form. I just didn't see that kind of universal and flexible and successful architectures before. I assume that it exists in some internal projects.

Kubernetes has strictly defined scope and even reduces that over time. It does not do anything with containers or images, it delegates it to CRI plugin with several popular implementations (containerd and crio). It does not do anything with networking, it delegates it to CNI plugin with multiple popular implementations (flannel, calico, cilium just to name a few). It does not do anything to interact with cloud platforms like AWS, etc. It has some conventions about cloud controllers and cloud vendors implement those controllers, providing integration, scaling and other cloud-specific features.

At the same time I'm using Kubernetes "directly". I don't write stuff which generates yamls. I write yamls directly. Sometimes I use templating engines, mostly kustomize, but those do not reduce much work. And deploying simple application requires quite a lot of YAML lines, most of those being boilerplate. For example docker-compose provides super-concise language. I don't need to write any boilerplate, I just write stuff that makes sense. With Kubernetes, I sometimes have a feeling like I'm writing an assembly code instead of compact high-level code.

I feel like Kubernetes was supposed to be used with some great templating, but all we got was helm and kustomize...

It's not a huge issue, but that's probably a main issue for me.

Other than that, I pretty much like Kubernetes and would use it everywhere for production. Even for single server. It has some learning curve, but once I learned it, I like it and wouldn't use, say, docker instead. And I used a lot of docker.

[KronisLV]

> At the same time its popularity makes it obvious that competition basically will not exist in the foreseeable future.

That'd be a bummer.

To me, Docker Swarm feels like the logical next step after single node Docker Compose deployments - supports networking and multiple nodes, while still using the simple Compose syntax. Sure, it's not as all encompassing as Kubernetes, but it feels like sometimes the latter isn't worth its complexity (if you want to self-host everything), outside of options like K3s.

I'd even say that something like Hashicorp Nomad is pretty nice if you don't have a problem with HCL. Plus, it supports not just containers, which is probably a boon to many: https://developer.hashicorp.com/nomad/docs/drivers

[thomster]

Whatever happened to swarm? I tried it maybe once, and that's that. Can't remember the last time I've seen docker swarm in a job ad also? Pretty much everything revolves with kubernetes or some combination with nomad.

[vbezhenar]

It is supported and probably works, but everyone just uses Kubernetes. If you're big, you want to use managed proven solution. If you're small, you expect to get big one day, and you want to use future-proof technologies to avoid unnecessary migrations. I think that's the way it works.

Hashicorp was bought by IBM, so I don't have high hopes for Nomad, either... IBM is a graveyard of good technologies.

[thomster]

oh man, just read it, too bad.....

[KronisLV]

I still use Docker Swarm and have seen it a few times, but yes, it’s definitely not common, which is a shame, because it also works pretty well with Portainer. Literally one of the most painless solutions run self-host and administer in my experience, at least for smaller scales.

[thomster]

I know that Docker like the company recently made a comeback (they had some financial issues), but never used Portainer, but they look cool...they remind me of Rancher....do they have a free version?

[KronisLV]

Yes, I’m using the Portainer community version myself, it also works with K8s so can serve as a lightweight alternative to Rancher as well, except last I checked there was a little bit of weirdness around non standard ingress controllers like Traefik instead of Nginx.

[thomster]

me too, that's kind of my point, once you get a feel for it, and understand it, you're set.