[pm90]

> It's common to see a 10x reduction in LOCs going from CloudFormation to Terraform and a 10x reduction further going from Terraform to Pulumi.

I don't see this as such a terrible problem. The configurations may have more LOC's but there are not as many surprises. The dependency of declarable configuration makes it rock solid and favorable among operations teams who need to make these kinds of changes all the time.

> A key importance in how Pulumi works is that everything centers around the declarative goal state. You are shown previews of this (graphically in the CLI, you can serialize that as a plan, you always have full diffs of what the tool is doing and has done. This helps to avoid some of the "danger" of having a turing-complete language. Plus, I prefer having a familiar language with familiar control constructs, rather than learning a proprietary language that the industry generally isn't supporting or aware of (schools teach Python -- they don't teach HCL).

I understand the reason to want this. Having worked closely with developers, lack of familiarity with HCL makes it much less accessible. However, from an operations perspective, I am GLAD that HCL is a very limited language. No imports of libraries all over the place (in your infrastructure configurations, no less!).

[throwaway894345]

> I don't see this as such a terrible problem. The configurations may have more LOC's but there are not as many surprises. The dependency of declarable configuration makes it rock solid and favorable among operations teams who need to make these kinds of changes all the time.

The issue is that your static configs often have lots of boilerplate sections that have to be kept in sync. Further, you can use an imperative language like Python, JS, etc and still write in a completely declarative fashion (or you can use a functional language which tend to be declarative out of the box). Conversely, you can model an AST in YAML (which is what CloudFormation is trending toward) and get the worst of all worlds. Bottom line: don't conflate "reusability" with "imperative" or "static" with "declarative".

[pm90]

> The issue is that your static configs often have lots of boilerplate sections that have to be kept in sync.

Yes, I agree with this. However, its predictable. As an operations person, I value predictability and am willing to pay the price of keeping static configs in sync.

> Further, you can use an imperative language like Python, JS, etc and still write in a completely declarative fashion (or you can use a functional language which tend to be declarative out of the box). Conversely, you can model an AST in YAML (which is what CloudFormation is trending toward) and get the worst of all worlds. Bottom line: don't conflate "reusability" with "imperative" or "static" with "declarative".

Hold on, I'm not conflating anything. Saying that "you can write terrible things in any language" isn't anything new. We choose to use languages that provide certain guarantees that we need for the domain that we're working in. For infrastructure, declarative languages are a lot more suitable for the properties they provide (i.e. no surprises, limited functionality etc.). Its "possible" to use static types in Python, how many do that?

[throwaway894345]

> Yes, I agree with this. However, its predictable. As an operations person, I value predictability and am willing to pay the price of keeping static configs in sync.

I think there's wisdom in this at small scales, but as the volume and complexity of your boilerplate grows, I think you lose any advantages. I also think this threshold is quite low (as an ops person and a dev person) since it's not much harder to look at/read the YAML generated by a script vs that which is hand-rolled and committed to git.

> Hold on, I'm not conflating anything.

Are you sure? Because you just said "I am willing to pay the price of keeping static configs sync" and then "For infrastructure, declarative languages are a lot more suitable for the properties they provide" and then you started to talk about "static types" in Python, which is different than "static" in the YAML sense (YAML isn't statically typed, but it is static in that it isn't evaluated or executed).

I'm not trying to be a jerk, it just sounds like a lot of concepts are being confused. I also wasn't making the argument "you can write terrible things in any language" (not sure if you were attributing that argument to me or if that was a point you were trying to make).

Consider this Python: https://github.com/weberc2/nimbus/blob/master/examples/src/n...

It's fully declarative, but it does evaluate, so it's not static in the YAML sense. It outputs a JSON CloudFormation template (but it could easily output in YAML) which you could inspect visually before passing onto CloudFormation.

It's also statically typed although that's not evident from this file since all types are inferred in this file (however there are annotations in the imported libraries), and while the static typing is a very useful property, it's not what I've been talking about in this thread.

In my opinion, this is no less readable than the equivalent YAML; however, it's capable of doing much more (albeit if your infrastructure is just one S3 bucket, then this is overkill--to really understand the power of dynamic configuration, you would want a more complex example).

[FridgeSeal]

I can’t trust my teammates to write code that doesn’t use raw eval()’s all over the place.

Getting them, nevermind relying on them to write Python/JS in the correct way is straight up out of the question.

At least I know in Terraform/HCL they can’t map a config change over the 1000 new instances they spun up because they happened to write their for loop wrong.

[throwaway894345]

> I can’t trust my teammates to write code that doesn’t use raw eval()’s all over the place. Getting them, nevermind relying on them to write Python/JS in the correct way is straight up out of the question.

Then use Starlark (https://go.starlark.net) or Dhall or similar.

> At least I know in Terraform/HCL they can’t map a config change over the 1000 new instances they spun up because they happened to write their for loop wrong.

To be clear, the proposal is to use a programming language to generate your HCL-equivalent configs, not to imperatively modify infrastructure. Consequently, you can inspect the generated "HCL" (or whatever the output is) and make sure it looks like the code they would write manually. Further, you can even write automated tests.

[cryptonector]

So, things need to be comprehensible by the humans that work with them. A 10x reduction in LoC / 10x increase in expressibility may or may not be a good thing, but if it captures intent better and with less ceremony and cruft, then it most decidedly is a FANTASTIC thing. Whereas a 10x LoC improvement that makes it harder to glean intent would be DISASTROUS.

Then again, code has to be run in order to analyze its output -- that or code has to be data you can analyze (like a Lisp), but that can be very difficult to reason about.

So my preference would be to have libraries for constructing configuration data. Then you can execute a program to generate the configuration, and that you can use without further ado. The output may not be easy for a human to understand, though it should be possible to write code to analyze it.