Hacker News new | ask | show | jobs
by ithkuil 924 days ago
A few years I go I tried out an alternative approach to "templating".

Basically the idea starts from a world without templates where you would distribute the k8s YAML in a form that is ready to be directly applied, with whatever sensible defaults you want directly present in the YAML

The the user would then just change the values in their copy of the file to suit their needs and apply that.

We all recoil in horror to such a thought, but let's stop a moment to think about why we do:

The user effectively "forked" the YAML by placing their values there and what a nightmare would that be once the user would get a new version of the upstream file, potentially completely overhauled .

If the changes are very small, a simple three way merge like you'd do with git would suffice to handle that. But what about larger changes?

Most of the conflicts in the simple cases stem from the fact that text based diff/merge tools are oblivious to the structure of the YAML file and can only so a so-so job with many of the changes. Unfortunately most people are familiar only with text based merge tools and so they have been primed the hard way to assume that the merges only rarely work.

Structural merges otoh so work much much better. But still if the upstream refractors the application in a significant way (e.g. changes a deployment into a stateful set or moves pieces of config from a configmap into a secret!) not even a structural merge can save you.

My idea was to bring the manifest author into play and make them "annotate" the pieces of the manifest forest that contain configuration that has a high level meaning to the application and that would be moved around in the YAML forest as it gets reshaped.

Another realization was that often such configuration snippets are deeply embedded in other internal "languages" wrapped inside string fields, subject to escaping and encodings (e.g. base64). E.g. a JSON snippet inside a TOML string value inside # base64 encoded annotation value (if you haven't seen these abominations I'm so happy for you you innocent child)

So I implemented a tool that uses neated bidirectional parsers ("lenses") that can perform in-place editing of structured files. The edits preserve formatting, comments, quoting styles, etc.

Even steing fields that are normally thought of as just strings are actually better though if as nested "formats". For example the OCI image references are composed of multiple parts. If you want to just copy images to your private registry and "rebase" all you image references to the new base, you can do it with an update that understands the format of the OCI image references instead of just doing substring replacement.

Knot8 is an opinionated tool meant to help manifest authors and users manage setting/diffing/pulling annotated YAML k8s manifest packages

https://github.com/mkmik/knot8

I didn't have the time to evangelize this approach much so it didn't get any traction (and perhaps it would because it doesn't have enough merits). But I encourage to give it a go. It might inspire you

I also pulled out the "lens" mechanism in a separate binary in case it could be useful to edit general purpose files:

https://github.com/kubecfg/lensed