[KaiserPro]

Distributed systems are hard, as well all know.

However the number of people that actually need a distributed system is pretty small. With the rise of kubernetes, the number of people who've not been burnt by going distributed when they didn't need to has rapidly dropped.

You go distributed either because you are desperate, or because you think it would be fun. K8s takes the fun out of most things.

Moreover, with machines suddenly getting vast IO improvements, the need for going distributed is much less than it was 10 years. (yes i know there is fault tolerance, but that adds another dimension of pain.)

[sd9]

> the number of people who've not been burnt by going distributed when they didn't need to has rapidly dropped

Gosh, this was hard to parse! I’m still not sure I’ve got it. Do you mean “kubernetes has caused more people to suffer due to going distributed unnecessarily”, or something else?

[boarush]

Had me confused for a second too, but I think it is the former that they meant.

K8s has unneeded complexity which is really not required at even decent enough scales, if you've put in enough effort to architect a solution that makes the right calls for your business.

[KaiserPro]

yeah sorry, double negatives.

People got burnt by kubernetes, and that pissed in the well of enthusiasm for experimenting with distributed systems

[DrFalkyn]

Because people, especially Devops, thought k8s was some magic, when all it really does is makes the mechanics easier

If you’re architecture is poor k8s won’t help you

[bormaj]

Any specific pitfalls to avoid with K8s? I've used it to some degree of success in a production environment, but I keep deployments relatively simple.

[KaiserPro]

Its a spectrum rather than a binary thing, however you are asking the right questions!

One of the things that is most powerful about K8s is that it gives you a lot of primitives to build things with. This is also its biggest drawback.

If you are running real physical infrastructure and want to run several hundreds of "services" (as in software, not k8s services) then kubernetes is probably a good fit, but you have a storage and secrets problem to solve as well.

On the cloud, unless you're using a managed service, its almost certainly easier to either use lambdas (for low traffic services) or one of the many managed docker hosting services they have.

Some of them are even K8s API compatible.

but _why_?

At its heart, k8s is a "run this thing here with these resources" system. AWS also does this, so duplicating it costs time and money. For most people the benefit of running ~20 services < 5 dbs and storage on k8s is negative. Its a steep learning curve, very large attack surface (You need ot secure the instance and then k8s permissions) and its an extra layer of things to maintain. For example, running a DB on k8s is perfoectly possible, and there are bunch of patterns you can follow. But you're on the hook for persistence, backup and recovery. managed DBs are more expensive to run, but they cost 0 engineer hours to implement.

BUT

You do get access to helm, which means that you can copypasta mostly working systems into your cluster. (but again like running untrusted docker images, thats not a great thing to do.)

The other thing to note is the networking scheme is badshit crazy and working with ipv6 is still tricky.