[husarcik]

The docker swarm ecosystem is very poor as far as tooling goes. You're better off using docker-compose (? maybe docker swarm) and then migrating to k3s if you need a cluster.

My docker swarm config files are nearly the same craziness as my k3s config files so I figured I might as well benefit from the tooling in Kubernetes.

Edit for more random thoughts: being able to use helm to deploy services helped me switch to k3s from swarm.

[amazingman]

This is almost exactly my experience with Docker Compose, which is lionized by commenters in nearly every Kubernetes thread I read on HN. It's great and super simple and easy ... until you want to wire multiple applications together, you want to preserve state across workload lifecycles for stateful applications, and/or you need to stand up multiple configurations of the same application. The more you want to run applications that are part of a distributed system, the uglier your compose files get. Indeed, the original elegant Docker Compose syntax just couldn't do a bunch of things and had to be extended.

IMO a sufficiently advanced Docker Compose stack is not appreciably simpler than the Kubernetes manifests would be, and you don't get the benefits of Kubernetes' objects and their controllers because Docker Compose is basically just stringing low-level concepts together with light automation.

[wg0]

Then the Helm and layers of kustomize are not easy to reason with either.

That's system configuration and that'll become tedious for sure.

[doctorpangloss]

Helm and Kustomize are low-budget custom resource definitions. They serve their purpose well and they have few limitations considering how much they can achieve before you write your own controllers.

In my opinion, the complexity is symptomatic of success: once you make a piece of some kind of seemingly narrowly focused software that people actually use, you wind up also creating a platform, if not a platform-of-platforms, in order to satisfy growth. Kubernetes can scale for that business case in ways Docker Swarm, ELB, etc. do not.

Is system configuration avoidable? In order to use AWS, you have to know how a VPC works. That is the worst kind of configuration. I suppose you can ignore that stuff for a very long time, you'll be paying ridiculous amounts of money for the privilege - almost the same in bandwidth costs, transiting NAT gateways and all your load balancers, whatever mistakes you made, as you do in compute usage. Once you learn that bullshit, you know, Kubernetes isn't so tedious after all.