[jpgvm]

Helm makes me sad.

What I do to remediate this sadness is use Helm from Tanka. There is still sadness but now it's wrapped in a nice Jsonnet wrapper and I can easily mutate the output using Jsonnet features without having to mess with nasty Go templating.

I've said it a million times before but it's always worth saying again:

Don't use string templating for structured data.

[ivan4th]

Yep. Many complain that with Lisp, you need to count parentheses (spoiler: you don't need to). And then proceed to count spaces for indent/nindent in the charts... That's somehow ok with almost everyone

[morelisp]

It's absolutely crazy to me how many tools are in common use for k8s templating which would all be wiped away with any decent macro system.

[twelfthnight]

I can't actually put it into production at my company, but for selfish catharsis, I ran datamodel-codegen over our cluster's jsonschema and generated Python pydantic models for all resources. I was able to rewrite all our helm using pure Python and Pydantic models, since Pydantic serializes to json and json is valid yaml. Felt pretty good

We don't have any CRD, but the approach would extend to those, plus you get auto complete. The k8s jsonachema isn't super easy to work directly with, though.

[speedgoose]

The template engine is not specific to Kubernetes but Golang. I wish they used something more adapted. https://pkg.go.dev/text/template

[morelisp]

Not just helm. There are probably a half dozen tools for rendering manifests in our company, only some use text/template, and they all suck. Text replacements are bad. Declarative structured patches are bad. Control flow in JSON is bad. We've had a language for dealing with generating complex nested structured data for years!

[ianburrell]

Have you seen Jsonnet, Dhall, and Cue? They are configuration language that are more limited than general purpose languages, more powerful that static, and designed for config files unlike templates.

[anonacct37]

text/template is probably ok... For some version of text. ditto with jinja and most templating languages. The cardinal sin of DevOps is using text macros to produce structured data. It only exists because unfortunately there is no other lowest common denominator for every config file syntax.

[morelisp]

Sure and that forgives its use in maybe, like, Salt and Ansible. Not in Kubernetes where everything is structured in the same way, even with API-available schemas, to begin with.

[mountainriver]

Jsonnet isn’t great either, and has been tried a bunch in the k8s community.

I’ll never understand why we don’t just use a language. I started writing all my k8s config in python and it’s great.

[ithkuil]

Jsonnet wouldn't be as bad as it is if there was just a modicum of debugging aid.

I'm slowly chipping away at that problem by implementing some tooling. For example I recently added "traceback" functionality in https://github.com/kubecfg/kubecfg

Another thing that I noticed is that most people who end up writing template libraries for jsonnet are using too many functions and not leveraging the strengths of jsonnet, namely object extension.

I opensourced a library I'm using internally at $work. It's far from perfect and sorely lacking docs and examples but if you want to give jsonnet another go I'd recommend you try kubecfg + https://github.com/kubecfg/k8s-libsonnet

[mplewis]

I agree. I write all of my K8s and surrounding cloud infra specs in Pulumi using TypeScript. Never going back to Helm.

[jpgvm]

The problem with Pulumi is that it wants to manage the state of things rather than letting the k8s server handle that. This means it's a hell of a lot slower than the equivalent `kubectl apply`. Not to mention it's own state persistence really sucks if you don't use their hosted solution that supports a patched based state update protocol (which someone really should implement an OSS version of).

I have be working on a new project and I split the IaC stuff into two layers, essentially using Pulumi (w/Kotlin) to spin up the k8s cluster and dependencies for Config Connector (on GCP). From there I'm just generating and applying manifests with fabric8 (more Kotlin).

It's not quite as good as Jsonnet in some cases (because of lazy vs non-lazy mostly and always-supported deep-merge etc) but Kotlin is immensely powerful and has things like the `lazy` helper to help here.

Having the entire repo defined in Kotlin though is very nice. Build system is Gradle w/Kotlin script, frontend is htmx only generated by Kotlin DSL, IaC all Kotlin as described.

We did similar at $CURRENT_JOB with Typescript but Kotlin is miles better IMO.

[jpgvm]

I will agree that it's not optimal but there is still a big difference between text templating and Jsonnet.

Something Jsonnet-esque but with more typing help, better debugging (especially error messages instead of `blah thunk, thunk thunk` would go a long way.