[corwin7]

If you can't edit a yaml (or HCL) file, you shouldn't be using Kubernetes / Terraform. All these "you too can be an artist, just draw inside the lines of this coloring book" configuration systems (ie helm) are just distracting people from doing things the right way.

[londons_explore]

Plain yaml files have far too much repetition for common use cases.

If I have 100 microservices, and I want to run all of them in a production and staging cluster, but I want all the staging cluster jobs to have an extra environment variable/tag and have lower CPU limits, there's no way to do that without 200 yaml files, which quickly get out of sync and inconsistent.

[atombender]

That's pretty extreme. What if you have a deployment that should have a 100GB PVC in production, but only 10GB in staging? Do you duplicate the entire config, or pass around patches, just to make a single line change? What if you have different HPAs, different replica counts, different resource requests/limits, etc.?

People use tools like Helm and Ksonnet because they allow you to write general-purpose manifests that can be parameterized. It's an extremely powerful tool. Our charts are full of things like this for that reason:

    resources:
      {{ if .Values.kubernetes.resources }}
      requests:
        {{ if .Values.kubernetes.resources.requests.cpu }}cpu: {{ .Values.kubernetes.resources.requests.cpu }}{{ end }}
        {{ if .Values.kubernetes.resources.requests.memory }}memory: {{ .Values.kubernetes.resources.requests.memory }}{{ end }}
      limits:
        {{ if .Values.kubernetes.resources.limits.cpu }}cpu: {{ .Values.kubernetes.resources.limits.cpu }}{{ end }}
        {{ if .Values.kubernetes.resources.limits.memory }}memory: {{ .Values.kubernetes.resources.limits.memory }}{{ end }}
      {{ end }}

Or stuff like dynamically built ingresses:

    - host: {{ .Values.host }}
      http:
        paths:
        {{ range .Values.apps }}
        - path: {{ .path }}
          backend:
            serviceName: {{ .name }}
            servicePort: 80
        {{ end }}

Admittedly, templating sucks, and a tool like Skycfg or Jsonnet would be significantly better here, but the fundamental problem to solve is the same.

I'm a big fan of configurator generators. I think it's wise to let apps be simple and rely o static, pre-baked configs (JSON, YAML, whatever), but a deployment system needs to be able to generate dynamic ones.

[dilyevsky]

Have you used terraform kubernetes provider? We (Cruise) basically gave up on it. Customer support reply about issues going ~1y or longer unfixed was “not a priority”.