[default-kramer]

I'm very surprised we don't see more people using a level 5 language to generate Terraform (as level 3 JSON) for this exact reason. It would seem to be the best of both worlds -- use the powerful language to enforce consistency and correctness while still being able to read and diff the simple output to gain understanding. In this hypothetical workflow, Terraform constructs like variables and modules would not be used; they would be replaced by their counterparts in the level 5 language.

https://developer.hashicorp.com/terraform/language/syntax/js...

[JanMa]

That actually works quite well. I once built a templating engine for Terraform files based on JQ that reads in higher level Yaml definitions of the resources that should be created and outputs valid Terraform Json config. The main reason back then was that you couldn't dynamically create Terraform provider definitions in Terraform itself.

Later on I migrated the solution to Terramate which made it a lot more maintainable because you write HCL to template Terraform config instead of JQ filters.

[rattyJ2]

This is basically how pulumi and tfsdk work. You write go/ts/python/... that generates a bunch of config files to a temp folder, and then reconciles those.

[harshitaneja]

This is exactly how we do it with our very own rudimentary internal library and scripts. Barely enough and even though I worry at times it will break unexpectedly but so far we are surprised by how stable everything has been.

Although I really wish there was a first party solution or a well established library for this but I suspect that while it is easy to build only enough to support specific usecases but building a generic enough solution for everyone would be quite an undertaking.