[otterley]

> Of course you're now maintaining the equivalent of github, circleci, S3, ....

On this we agree. That's a nontrivial amount of undifferentiated heavy lifting--and none is a core feature of K8S. You are absolutely right that you can use K8S CRDs to use K8S as the control plane and reduce the number of idioms you have to think about, but the dirty details are in the data plane.

[eddythompson80]

Yeah, but you significantly increase your changes of getting the data plane working if you are always using the same control plane. The control plane is setting up an S3 bucket for you. That bucket could come from AWS, CubeFS, Backblaze, you don't care. S3 is a simple protocol but same goes for more complex ones.

> and none is a core feature of K8S

The core feature of k8s is "container orchestration" which is extremely broad. Whatever you can run by orchestrating containers which is everything. The other core feature is extensibility and abstraction. So to me CRDs are as core to kubernetes as anything else really. They are such a simple concept, that custom vs built-in is only a matter of availability and quality sometimes.

> That's a nontrivial amount of undifferentiated heavy lifting

Yes it is. Like I said, the benefit of kubernetes is it gives you the choice of where you wanna draw that line. Running and maintaining GitHub, CircleCI and S3 is a "nontrivial amount of undifferentiated heavy lifting" to you. The equation might be different to another business or organization. There is a popular "anti-corporation, pro big-government" sentemnt on the internet today, right? would it make sense for say an organization like the EU to take hard dependency on GitHub or CircleCI? or should they contract OVH and run their own Github, CircleCI instances?

People always complain about vendor-lock in, closed source services, bait and switch with services, etc. with Kubernetes, you get to choose what your anxieties are, and manage them yourself.

[otterley]

> You significantly increase your changes of getting the data plane working if you are always using the same control plane.

That is 100% not true and why different foundational services have (often vastly) different control planes. The Kubernetes control plane is very good for a lot of things, but not everything.

> People always complain about vendor-lock in, closed source services, bait and switch with services, etc. with Kubernetes, you get to choose what your anxieties are, and manage them yourself.

There is no such thing as zero switching costs (even if you are 100% on premise). Using Kubernetes can help reduce some of it, but you can't take a mature complicated stack running on AWS in EKS and port it to AKS or GKE or vice versa without a significant amount of effort.

[eddythompson80]

Well you know, we went from not knowing that kubernetes can orchestrate everything, to arguing "k8s best practices" for portability so there is room for progress.

The reality is yes, noting is zero switching costs. There are plenty of best practices to how to utilize k8s for least headache migrations. It's very doable and I see it all done all the time.