[yebyen]

OK, fair. It's still worth a look! That was my point. If you compare it to other alternatives and find it to be the best thing, it would be a shame to put it on the shelf instead when there are no alternatives that are as technically strong.

If you don't look at it, you have no basis to compare it to the alternatives that you're evaluating. I would rather pick a really good solution than a supported one. If I have to pick a supported solution, then I would rather at minimum compare it to a good one, so I can be honest about what shortcomings it has. It is telling that most of the recommendations in this thread are Helm.

Helm is great, but it ain't no Deis.

Deis Workflow is rock-solid and worth evaluating, even if support is a requirement for you. Other competing solutions are not as good. I don't even know any other that are really comparable. Maybe OpenShift, but it is not "really K8S"

[xkarga00]

> Maybe OpenShift, but it is not "really K8S"

Can you elaborate more? What makes you think that this is true?

[yebyen]

It has incompatible resource types. They built OpenShift to handle authorization and permissions before Kubernetes RBAC was fully baked. So there are OpenShift solutions that don't exist on K8S, and vice-versa.

The BuildConfig and ImageStream for starters.

It's not a substantive difference that makes OpenShift much harder to learn, but it is a difference that means "if RedHat decides to 'Deis' OpenShift," we're stuck rebuilding everything for the better-supported K8S-proper mainline tree, and large parts of our tooling are going to need to be replaced because Kubernetes does not do ImageStream, and OpenShift does not do Helm.

Maybe the chances of that happening are low, but there are enough differences that from my understanding, I should not ever expect Kubernetes projects to be directly portable to OpenShift without modification (or vice-versa.)

It's also very expensive for an open-source project. Granted you are paying for support and cloud hardware, but I can take Kubernetes and spin it up anywhere. Try installing RedHat OpenShift onto arbitrary releases of CoreOS, Debian, and Amazon Linux like you might be able to do with kubeadm or kops.

That was one of the core promises of Kubernetes, to run anywhere that you can run Docker. My experiences with OpenShift were anything but that. (If I want to run OpenShift Origin, I'll be setting up a latest release of Fedora or RHEL to do it on, I guess.)

I will take Deis for my dev environments at least, because I think the chances that Kubernetes core devs are going to break the APIs in a way that makes it impossible for Deis to be kept alive by a ragtag bunch that figured out how the CI scripts work, pretty much nil. I can take Deis to any cloud provider on any operating system that can do Kubernetes, or onto my own metal (or on Minikube, or on Localkube, or ...)

You get the point by now... Kubernetes brings an ecosystem of options, and OpenShift narrows the scope and range of that ecosystem substantially.

When Deis v1 was EOL'ed, I got into a bit of an argument on HN with Bacongobbler about whether Deis v2 was a different product or not. I argued that it was, because it runs on a different platform now (K8S) and does not support running on the old platform anymore (Fleet).

Technically not true because you can run K8S on Fleet, and Deis v2 on that. But for a sysadmin, it was different, because I knew the rules about making Deis v1 with Fleet "High-Availability" and the rules were all different for Kubernetes, so I argued that it was different.

But for a user, the APIs are all compatible, and they may even bring API integrations such as deisdash.com with them to the "new" platform. (Deisdash is the only API integration for Deis that I am aware of, but when Deis v1 was EOL and Deis v2 was production-ready, you could use Deisdash with either.)

I've now fully eschewed Deis v1 (my old place of employment still has one standing, but it runs such a small amount of infrastructure that I could replace it with a severely less sophisticated setup and nobody would notice until it failed.) I'm on Workflow now, and I have approximately no regrets about it. I can take it anywhere that I can take helm and K8s.

I'll be looking forward to see what the Deis/Azure team bring out in the future that's going to obviate the need for me to be on unsupported EOL Workflow. Because according to Deis team lead @gabrtv, they are still just getting warmed up:

https://twitter.com/gabrtv/status/891096179089342464

[smarterclayton]

> Maybe the chances of that happening are low, but there are enough differences that from my understanding, I should not ever expect Kubernetes projects to be directly portable to OpenShift without modification (or vice-versa.)

It's true that openshift goes a lot further to disabling things that are dangerous or not ready. Ie preventing root containers, or not enabling third party resources until it went to beta. But everything that runs on Kube runs on openshift that depends on a beta feature or higher.

Re: other OSes - a large part of what we do at Red Hat is making all the other stuff work - Docker, filesystems, selinux, security, NFS, volume drivers, network, etc. A lot of times it's not worth the extra effort to track five distributions of anything, but instead to focus on making something actually work. The behind the scenes work outside of Kubernetes is just as important as the Go code, and so we focus on those few operating systems and making it all work together.

[yebyen]

The fact is, most Kubernetes projects I know are installed by Helm, and (it might have been you, personally who) explained to me that Helm is incompatible with a multitenant environment. I think they've made some strides since RBAC has gotten a little more polished, ... but please correct me if I'm wrong, OpenShift permissions model and RBAC are more compatible than I think.

The last I heard, you just can't really use Helm on OpenShift unless you go to some lengths to lock it down to a single namespace.

It would be amazing if someone could publish a Helm on OpenShift guide! Hmm, it seems you maybe already did: https://github.com/kubernetes/helm/issues/2517

[smarterclayton]

Helm isn't incompatible, it's just not currently set up for dealing with different tenants. You can use Helm in a single tenant fashion on OpenShift just like you can use it in a single tenant fashion on Kube today.

Starting with OpenShift 3.6 (on Kube 1.6) all RBAC roles between Kube and OpenShift are treated equivalently, and from OpenShift 3.7 onwards the OpenShift RBAC rules are just a compatible API shim on top of Kube RBAC. The out of the box rules on OpenShift are more restrictive simply to ensure that full multi-tenancy is possible, but they can always be lifted.

[yebyen]

Awesome. This makes me feel more optimistic about OpenShift, especially given that I probably can't realistically take Deis Workflow to production now.

(I don't know how much you've looked at Deis, but I couldn't think of anything better to compare it to than OpenShift. I could probably switch from Deis to OpenShift without too much hassle. Now I'm going to have to go ahead and try Deis _ON_ OpenShift, though :)

[yebyen]

Hmm. It looks like[1] I'm wrong about one more thing, that being where you can run OpenShift. The current installation docs describe a single all-in-one binary that you can use to run OpenShift origin on any current Linux kernel, or Mac:

[1]: https://docs.openshift.org/latest/getting_started/administra...

I don't know if you need to have Docker for Mac installed, but I would guess you don't (it would be crazy to try to interface with arbitrary versions of Docker, it probably runs its own docker inside of a virtualized layer.)

I'm going to have to look at this again in some more depth. Looks better than when I saw it last time! (That's to be expected, I guess, but again it is encouraging!)