[013a]

I would argue that, while Stripe is going with a scratch build, this could be motived by AWS's lack of a good managed Kube offering, which is changing in the next few months.

With a managed Kube offering, setting up Kube is much much easier than this jenkins setup you are suggesting. And, there's no overhead charge. Why would anyone go through the hassle of manually provisioning machines like you suggest when AWS/GCP will do it for you?

Its overkill in the same way using DynamoDB for something that only experiences a handful of writes every day is overkill; who cares? The scale is there if you need it, but it doesn't cost anything to not use it.

[kureikain]

Setting up a K8S cluster isn't that hard actually.

From my experience, the hard part kickin when dealing with stateful service which needs to associated with volume.

Even with a managed cluster, you still have to solve that problem. Either you pre-provision disk or use dynamic volume.

Next is when upgrading K8S version. with a stateless service, it's a walk in a prt to upgrade. With data volume it's more tricky to upgrade because you want to control the process of replacing node and want to ensure the data volume get mounted and migrated to new node properly.

Thing get harder especially with stuff like Kafka/ZooKeeper when pods get remove and the re-balancing happen.

In other words, managed Kuber actually offer not much. You still have to be carefully planning and it isn't magically solve all problem for you.

[013a]

That's true, but I'm not sure if using Jenkins would avoid these problems you outline. And that's really the crux of what the OP is suggesting; that Jenkins or something smaller than Kube would have been a better choice.

[kureikain]

That's a fair point. I agree that Jenkins will not solve these problem and in fact they come with their own problem anyway. I was argued on the sole point of setting up K8S.

Other than that I agree with you.

[thesandlord]

(I work for GCP)

For 95% of people, I'd say going with the managed version is the right choice.

However there are some reasons why you wouldn't use a managed service. If you need a custom build, custom drivers, etc.

[karussell]

Do you think GKE will support multi clouds setups or hybrid scenarios at some point? For cost reasons we have to put some big servers off the cloud ...

[user5994461]

And most importantly, to use across multiple clouds!