| From what I've seen in other projects, I think that translates to: 1. we have a management k8s cluster where we deploy app blueprints 2. the app blueprints contain, among other things, specifications for VMs to allocate, which get allocated through an OpenStack CRD controller 3. and those VMs then get provisioned as k8s nodes, forming isolated k8s clusters (probably themselves exposed as resource manifests by the CRD controller on the management cluster); 4. where those k8s nodes can then have "namespaced" (in the Linux kernel namespaces sense) k8s resource manifests bound to them 5. which, through another CRD controller on the management cluster and a paired CRD agent controller on in the isolated cluster, causes equivalent regular resource manifests to be created in the isolated cluster 6. ...which can then do whatever arbitrary things k8s resource manifests can do. (After all, these manifests might even include deployments of arbitrary other CRD controllers, for other manifests to rely upon.) All said, it's not actually that braindead of an architecture. You might better think of it as "k8s, with OpenStack serving as its 'Container Compute-Cluster Interface' driver for allocating new nodes/node pools for itself" (the same way that k8s has Container Storage Interface drivers.) Except that 1. there isn't a "Container Compute-Cluster Interface" spec like the CSI spec, so this needs to be done ad-hoc right now; and 2. k8s doesn't have a good multi-tenant security story — so rather than the k8s nodes created in these VMs being part of the cluster that spawned them, their resources isolated from the management-layer resources at a policy level, instead, the created nodes are formed into their own isolated clusters, with an isolated resource-set, and some kind of out-of-band resource replication and rewriting to allow for "passive" resources in the management cluster that control "active" resources in the sandboxed clusters. |