+1 to Pulumi. I can't imagine operating k8s without tools like Pulumi or Terraform now. And Pulumi addresses some of the most important shortcomings of Terraform so I really hope they gain more traction.
> we can then create functions where we would normally duplicate Yaml.
This is the way to go for sure. I've done similar by generating CloudFormation from Python (I wrote my own library because I felt Troposphere was not very friendly nor a significant improvement over YAML).
Typing turns out to be pretty useful when you're generating YAML. While my library was fully typed, Python's type checking left a lot to be desired--many stupidly common things still can't be expressed (JSON, kwarg callbacks, etc), getting mypy to find/accept type annotations for libraries is stupidly hard, and the IDE integrations are pretty awful at this point. TypeScript users would enjoy a real leg-up here since its type system isn't half baked.
> This is the way to go for sure. I've done similar by generating CloudFormation from Python (I wrote my own library because I felt Troposphere was not very friendly nor a significant improvement over YAML).
Yes and no.
Typing is a must, but a full-blown programming language is too powerful and all abstraction layers start to leak sooner rather than later. I always ended up with a "deployment" function that exposed almost all underlying functionality.
It depends. If you can reasonably trust your team to not to do I/O in the scripts, then you're fine. If you can't, then you should use something like Starlark (a subset of Python running in a constrained environment that precludes I/O, unbounded loops, etc); however, Starlark doesn't support types yet.
I've looked at Cue a few times (principally out of frustration with the lack of types in Starlark), but I don't really "get it". What I want is a typed embeddable scripting language--specifically the ability to expressively generate YAML, and I'm pretty sure that's not what Cue is. I'm open to the argument that Cue is better than what I want (a la "if Henry Ford asked people what they really want, they'd've said faster horses"), but the value proposition isn't clear to me from perusing the docs. Maybe you can correct me?
I have used a bunch of jsonnet and was in your position. I picked up https://jkcfg.github.io and have been very happy. You get TypeScript with (some) hermeticity guarantees.
Are you not worried about people writing arbitrary code to do stuff? I've been burnt before where devs used Turing complete languages (python in my case) to generate configs in probably the most convoluted and complicated manner possible. It was impossible to debug and understand, there were side-effects literally everywhere. It was everything you'd imagine from a normal bit of bad code, but it also happened to spin up hardware.
Pulumi looks good, but it confuses me. I thought it was like Ansible/Terraform, but I see a "pricing" page and it looks like there's a SaaS that goes with it, can someone shed some light?