[sszuecs]

Running >100 Kuberenetes clusters with a team is already quite complex. Teams already struggle with Kubernetes core concepts and we have to make sure the time spent does not grow, because of some nice to have features that are not business relevant. Adding more complexity adds a lot of costs and most of the people looking into features like istio might see that it provides less value as promised by the advertisements. We use instead of service mesh is simple and flexible ingress controller, which can be used as api gateway https://opensource.zalando.com/skipper/. With simple annotations (core Kubernetes concept) we can do blue/green deployments, A/B tests, shadow traffic, you can build complex http routing as you wish and change everything in http in the request or response path. IMHO there is no much value left that service meshes provide: mtls and network policy maybe, everything else skipper provides. Blue/green with a simple kubectl plugin: https://github.com/szuecs/kubectl-plugins More advanced automated blue green deployments via https://github.com/zalando-incubator/stackset-controller. Skipper also supports several openapi providers and is used by several production users.

We add dns names for cluster internal communication through coredns templates to this api gateway. https://opensource.zalando.com/skipper/kubernetes/east-west-... You can adapt it step by step and stop without a bid all in approach as service mesh vendors promote.

If you have not enough you can get even more advanced and add hpa by requests per second.

We use all of these in >50 production clusters with regular shop and order traffic far beyond 10k requests a second. We are interested in features, but even more in stability. We are interested in the community and will fix most of the issues you will show us to make errors less likely happen for us, too. We don’t sell software nor support, that’s why we have less advertisements and don’t show up on every kubecon.

I hope to read you in our community channels: https://github.com/zalando/skipper/blob/master/readme.md#com...