[fancythat]

I am going to state a somewhat unpopular opinion, Kubernetes in their current form are just short from unusable in complex production environment. In order to run a full Kubernetes stack, you need to get a multitude of components running, each with own name with unknown effects on the underlying system.

I have experiences with two forms of deployment: one was an on-prem installation two years ago, on which one of the 13 Java 8 applications had very large latencies when accessing Oracle DB, otherwise working fine when it was deployed on simple VM. All of those applications had been done with the same DB logic, and we couldn't find the issue on our own, so we asked third-party to debug this issue for us: they couldn't pin point the problem, even with commercial tools. Their answer was just, something is off with your Kubernetes installation and that was it.

My second, on-going experience, is my current assignment with Fortune 500 company, that uses GKE for running hundreds of nodes, after migrating from on-prem VMs. Almost every other week (99% reliability - yeah right), some part of the system just dies and leaves services unreachable or unresponsive. There is a continuous effort to solve this issues and even Google support was contacted with the answer boiling down to: shit happens, deal with it. The only solution in those situations is either to have alarms go off so that Ops can restart something, or just to wait until everything comes back up again on its own.

The whole ecosystem was a good idea that lacks proper tool and stability to provide substantial benefits over the bunch of VMs, IMO.

[citrin_ru]

I have an impression that K8s to some extent benefits from a positive feedback loop - developers/ops advocate it usage to learn and put it on their CV (even if otherwise it is a bad choose), managers choose it thinking that it would be easy to hire people with K8s experience given that a lot of people around trying to learn it. I could not tell about my experience with K8s (NDA) but it doesn't look pretty to me too.

[fancythat]

It is definitely a new thing that is highly sought after and I understand people that are pursuing it. However, if you need to make a decision if this is a usable tool for your setup, please, invest some time to test everything possible before moving to it.

[bzzzt]

> Almost every other week (99% reliability - yeah right), some part of the system just dies and leaves services unreachable or unresponsive. There is a continuous effort to solve this issues and even Google support was contacted with the answer boiling down to: shit happens, deal with it.

The entire point of Kubernetes is redundancy through multiple stateless service instances that can and will be killed at any moment. If you take an application that doesn't work in such an environment, for instance a highly stateful application, that will cause pain. If you want simple 'lift and shift' to the cloud avoid Kubernetes.

[fancythat]

Applications in question are all stateless and worked in distributed VM environment without issues, usually with simple Nginx or HAProxy redundancy setup. I see your point and it is valid, however, this wasn't the case in the examples I mentioned.