[dijit]

I got lazy and just wrote scripts that output k8s manifests.

The development story is much better (breakpoints! WHAT!?, loops and control flow!?), you can catch common issues quicker by adding tests, there's one "serialise" step so you don't have to deal with YAML's quirks and you can version/diff your generated manifests.

It's dumb, and stupid, but it works and it's far less cognitive load.

Now: handling mildly dynamic content outside of those generated manifests... that's a massive pain, releasing a new version of a container and avoiding to touch the generated manifests: not working for me.

[temp_praneshp]

at my current place, we started off with kustomize. I rewrote everything into helm, which was good initially (at least you can force inject some common params, and others can include this in their charts).

But people (including me) were unhappy at yaml reading; I also grew to hate it with a passion because it's neither go nor yaml, and super difficult to read in general. We are a typescript company, and https://cdk8s.io/ has been great for us. We can unit test parts of charts without rendering the whole thing, distribute canonical pod/deployment/service definitions, etc.

In all of the cases, we combined this with config outputted by terraform, for env specific overrides, etc.

[imglorp]

Found the workaround confession thread.

Because you effectively CAN'T dynamically configure subcharts with templating that's done in your main chart, see eg https://github.com/helm/helm/pull/6876 here comes the hack.

We run helm in helm. The top chart runs post-install and post-upgrade hook job which runs helm in a pod with a lot of permissions. The outer helm creates values override yaml for the subchart into a ConfigMap, using liberal templating, which gets mounted in the helm runner pod. Then helm runs in there with the custom values and does its own thing.

Not proud but it lets us do a lot of dynamic things straight helm can't.

[crabique]

Have you considered https://github.com/helmfile/helmfile ? What you described sounds a lot like what Helmfile does, but more terrifying.

[imglorp]

Yes the hack is gross and yes I would love to use Helmfile. But our use case is to fit into customers' helm processes, eg ArgoCD et al.

[leetrout]

I do the same with Terraform sometimes.

I appreciate that TF has loops and dynamic blocks, etc etc, but sometimes it's just a lot easier to look at a Jinja2 template and run a script to generate the TF.