| Same exact question I ask every single time. We just decided against k8s, again, in 2024. We are going to go with AWS ECS and Azure Container Apps (the infra has to exist in both clouds). ECS and Container Apps provides all the benefits of k8s without the cons. What we want is a to be able to execute container (Docker) images with autoscaling and control which group of instances can talk to each other. What we do not want to do: - learn all of the error modes of k8s - learn all the network modes of k8s - learn the tooling of k8s (and the pitfalls) - learn how to embed yaml into yaml the right way (I have seen some of the tools are doing this) - do upgrades of k8s and figuring out what has changed the way that is backward incompatible - learn how to manage certificates for k8s the right way - learn how to debug DNS issues in a distributed system (https://github.com/kubernetes/kubernetes/issues/110550 and many more) I could go on and on but many people and companies figured out the hard way that k8s complexity is not justified. |