|
K8s is complicated because it does many things, not because each of the things it does, individually, is complicated. If your orchestrator only runs stuff, if your service discovery is independent of the orchestrator, and just does the wiring and routing, if your secrets management is simply a thing that provides secrets to authorized clients, a lot of the complexity associated with kubernetes goes away. The whole is always more than the sum of its parts, and in the case of kubernetes this adds up in an unfavorable way, complexity-wise. IME, a lot of complexity, in kubernetes land, also comes from kubernetes being highly opinionated. Whenever your needs don't match kubernetes' opinions, you have to fight it. To give you an example, service versioning is not something kubernetes knows about. If you need it, you need to add it manually. You can easily automate this with consul, and make applications unaware of service versions. Reciprocal trust verification between ... things that communicate is another. Kubernetes provides nothing. If someone manages to inject a pod into your namespace, and randomly starts replaying requests, there's no automatic way to verify the caller's identity - you have to do this manually, at the application level, using a custom sidecar (if you're lucky, your service mesh has something to help you). If you use consul with nomad, you get these things out of the box. I believe your life is simple now because what you manage isn't overly complicated. Once you have thousands of distinct deployments, versioned, across several clusters, and thousands or even tens of thousands of pods running in tens of data centers, life with kubernetes becomes a lot more difficult. Of course, you can still manage, by automating everything, but automation, at that level, with kubernetes, is not an easy thing. Or at least that's my experience. |