[kstrauser]

> Picking Terraform over Cloudformation: Endorse

I, too, prefer McDonald's cheeseburgers to ground glass mixed with rusty nails. It's not so much that I love Terraform (spelled OpenTofu) as that it's far and away the least bad tool I've used in the space.

[orwin]

Terraform/openTofu is more than OK. The fact that you can use to to configure your Cisco products as well as AWS is honestly great for us. It's also a bit like ansible: if you don't manage it carefully and try to separate as much as possible early, it starts bloating, so you have to curate early.

Terragrunt is the only sane way to deploy terraform/openTofu in a professional environment though.

[kstrauser]

I curse at Terraform at least once a week, usually right after I’ve discovered some weird arbitrary limitation surprising misfeature. It’s still what I reach for when I need to manage a whole organization. And compared to CloudFormation, it’s the freaking Cistine Chapel of IaC.

[MrDarcy]

We can also use expect to configure Cisco routers and AWS infrastructure, doesn’t mean we should.

[xorcist]

I never understood this. Why not use Ansible instead, especially if you already use it? Doubly so when you have Cisco config to manage. The experience is generally so much better it's not comparable, and it is much easier to infer running state.

[gchamonlive]

Ansible and terraform have some overlap, but they do tend to serve different purposes. The consequences of terraform having a state file should steer your decision.

However, I often find ansible modules to be confusing to use. Maybe with LLMs it's now easier to draft ansible roles and maintain them, but I always had agro whenever I needed to go to the docs for something I've done many times just because the modules are that much inconsistent.

[xorcist]

Setting aside the turing completeness of them, in practice Ansible is a complete superset of Terraform. From experience, the only times you appreciate the state file is when you have uncontrolled changes, in which case you are in for a bad time anyway.

Ansible modules are trivial to write and more people should. Most are trivial in practice and just consists of a few underlying API calls. A dozen line snippet you fully understand is generally not a maintenance burden. A couple of thousand someone else wrote might be.

[orwin]

It's not the same purpose. Ansible is useful to configure your IAC, terraform to deploy and handle the state, which is very useful when you have multiple teams working on the same infrastructure.

What you can do if you _really_ like ansible is to use it to generate the terraform files (typically from Jinja2 template). In practice, i think Terragrunt is easier to use if you already have terraform modules. But if i was back at my first "real" job, where we had between 50 and 80 ansible modules (very short ones, it was really good, i've never saw an infrastructure that complex handled that concisely and easily), and if we had to use terraform, i would use ansible to generate terraform files 100%.

[OJFord]

Because one's a language & tool for infrastructure as code, and the other's essentially a low-code ops scripting framework?

[walt_grata]

I've been very happy using cdk for interacting with aws. Much better than terraform and the like.

[etothet]

I second this. I do use some terraform, but for most of our stacks, CDK has been fantastic.

[nine_k]

Any opinion on Pulumi?

[gouggoug]

Not an opinion on Pulumi specifically, but an opinion on using imperative programming languages for infrastructure configuration: don't do it. (This includes using things like CDKTF)

Infrastructure needs to be consistent, intuitive and reproducible. Imperative languages are too unconstrained. Particularly, they allow you to write code whose output is unpredictable (for example, it'd be easy to write code that creates a resources based on the current time of day...).

With infrastructure, you want predictability and reproducibility. You want to focus more on writing _what_ your infra should look like, less _how_ to get there.

[juffasdfwa]

The imperative trite just comes off as geriatric. There are better arguments you can use here which you have shared below. One of which I agree with.

[__turbobrew__]

> for example, it'd be easy to write code that creates a resources based on the current time of day

Some languages make that impossible to do, for example starlark.

[nothrabannosir]

Couldn't disagree more.

I have written both TF and then CDKTF extensively (!), and I am absolutely never going back to raw TF. TF vs CDKTF isn't declarative vs imperative, it's "anemic untyped slow feedback mess" vs "strong typesystem, expressive builtins and LSP". You can build things in CDKTF that are humanly intractable in raw TF and it requires far less discipline, not more, to keep it from becoming an unmaintainable mess. Having a typechecker for your providers is a "cannot unsee" experience. As is being able to use for loops and defining functions.

That being said, would I have preferred a CDKTF in Haskell, or a typed Nix dialect? Hell yes. CDKTF was awful, it was just the least bad thing around. Just like TF itself, in a way.

But I have little problems with HCL as a compilation target. Rich ecosystem and the abstractions seem sensible. Maybe that's Stockholm syndrome? Ironically, CDKTF has made me stop hating TF :)

Now that Hashicorp put the kibosh on CDKTF though, the question is: where next...

[Too]

This does not match my experience. Terraform validate will check for types and you have great IDE support with refactor, find usages, resource documentation and everything else you might expect. You can of course build a monster of untyped dictionaries but that’s true for almost any language. Do you have examples of something else that the validation step does not find?

[nothrabannosir]

I never had this integrated with an editor but maybe I didn’t try hard enough. With CDKTF, a stock typescript IDE automatically has tab complete for every provider you want. That + being able to use functions, variables, loops, arrays, and structs for composition made it a completely different experience.

If those things all end up coming to terraform I’m all ears. For now it seemed more like a compilation target.

[kstrauser]

Thanks for saving me the trouble of writing exactly that. I want my IaC to be roughly as Turing complete as JSOJ. It’s sooo tempting to say “if only I could write this part with a for loop…” and down that path lies madness.

There are things I think Terraform could do to improve its declarative specs without violating the spirit. Yet, I still prefer it as-is to any imperative alternatives.

[popalchemist]

yes. IaC is a misnomer. IaC implementations should have a spec (some kind of document) as the source of truth; not code.

[vanviegen]

> Particularly, they allow you to write code whose output is unpredictable

Is that an easy mistake to make and a hard one to recover from, in your experience?

The way you have to bend over backwards in Terraform just to instantiate a thing multiple times based on some data really annoys me..

[gouggoug]

> Is that an easy mistake to make and a hard one to recover from, in your experience?

If you're alone in a codebase? Probably not.

In a company with many contributors of varying degrees of competence (from your new grad to your incompetent senior staff), yes.

In large repositories, without extremely diligent reviewers, it's impossible to prevent developers from creating the most convoluted anti-patterny spaghetti code, that will get copy/pasted ad nauseam across your codebase.

Terraform as a tool and HCL as a programming language leave a lot to be desire (in hindsight only, because, let's be honest, it's been a boon for automation), but their constrained nature makes it easier to reign in the zealous junior developer who just discovered OOP and insists on trying it everywhere...

[juffasdfwa]

> but their constrained nature makes it easier to reign in the zealous junior developer who just discovered OOP and insists on trying it everywhere...

I don't think this is true anymore. Junior devs of today seem to be black pilled on OOP.

[gouggoug]

Let my geriatric self rephrase this for you and make the point more obvious: "[...] who just discovered [insert latest design pattern trend of your choice] and insists on trying it everywhere"

[franktankbank]

Agreed, I'm fine with a declarative format in one file as long as I can control the imperative bits on which it depends.

[awoimbee]

I was the sole DevOps at my company for a long time, the team is now bigger. I used terraform for AWS and pulumi for K8S (terraform was too restrictive).

IMO pulumi is a huge gain of productivity when you know what you're doing. Cons: * It's plagued by bugs and the pulumi-kubernetes provider is not getting enough attention from the pulumi team (they're always working on compatibility with yet another language instead of focusing on one thing) * You end up with your very specific/personal codebase instead of having a generic/standard thing

Still, no regrets, I saved so much time thanks to pulumi!

With a bigger team: * Oboarding people takes more time * You end up with code quality issues. Most "DevOps" people aren't devs, sadly.

[x3n0ph3n3]

My opinion is there are not enough good software developers doing DevOps, and HCL is simple enough and can have pretty good guardrails on it. My biggest concern is people shooting themselves in the foot because the static analysis tools available for HCL don't work with Pulumi.

[SlightlyLeftPad]

It’s an unfortunate truth that good software developers aren’t crazy enough to want to do it.

[orthecreedence]

Pulumi is superior to Terraform for my use cases. It's actually Infrastructure as Code. Terraform pretends to be, but uses a horrible config language that tries to skirt the line between programming language and config spec, and skirts it horribly. Reorganizing modules is a huge pain. I dreaded using Terraform and I spin things up and down in Pulumi all day. No contest.

Granted, I'm a programmer, have been for a long time, so using programming tools is a no brainer for me. If someone wants to manage infra but doesn't have programming skills, then learning the Terraform config language is a great idea. Just kidding, it's going to be just as confusing and obnoxious as learning the basic skills you need in python/js to get up and running with Pulumi.

[kstrauser]

I disagree with that. I think it’s satisfying to find a way to express my intent in HCL, and I don’t think I could do it as well without a strong programming background.

[sandorscribbles]

manage the infrastructure with infrastructure tools - manage the application with application tools. they are not the same thing. you do not need to change the oil on your cars seats...drivetrains and interiors are different worlds joining together to achieve the goal of moving humans around.

[AIorNot]

We used it at my last startup and I loved it but im a dev not devops guy

I loved reading code

[easterncalculus]

You can honestly do a lot of what people do with Terraform now just using Docker and Ansible. I'm surprised more people don't try to. Most clouds are supported, even private clouds and stuff like MAAS.

[x3n0ph3n3]

Yeah, but ansible is one of the nine circles of hell and its support for various AWS services beyond EC2 and S3 is near nonexistant.

[Tostino]

I have mixed feelings about it. On my first startup, I used ansible to automate all of the manual workflows and server setup that we had done. Everything was just completely manual and in people's heads before, and translating it to ansible was a pain in the ass to say the least. I don't think it would have been any easier to translate it to something else though. It ended up working fine and we had a solid system that I could reset up our environment from scratch on a set of VPS provided by some terraform scripts. We were originally on digitalocean, and had to migrate to Azure because of acquisition BS.

For my current startup I ended up not going a direction where I needed ansible. I've now got everything in helm charts and deployable to K8S clusters, and packaged with Dockerfiles. Not really missing ansible, but not exactly in love with K8S either. It works well enough I guess.

[SkiFire13]

> on a set of VPS provided by some terraform scripts

You ended up needing Terraform too for the infrastructure though. At that point why not just use Terraform?

[Tostino]

Terraform was just for interacting with the cloud provider and spinning up the servers. Ansible was responsible for deploying all dependencies and getting the servers actually ready for use. Remember, none of this architecture was dockerized.

I had originally used Ansible to interact with the cloud provider and do the provisioning too, but someone on the corporate infrastructure team wanted to use terraform for that instead, so they did the migration.

[easterncalculus]

I also have experience with using Terraform and Ansible like this. Once I realized that most of the work actually performing the setup beyond blank VM creation was in Ansible, I was much more interested in just using Ansible for everything.

[MrDarcy]

CDK is far better than Terraform.

[YetAnotherNick]

CDK is better when it works. Terraform has so many escape hatches it scales better with edge cases over time.

There are all sort of requirements that pops up, specially in times of downtime or testing infra migration in production etc. and it's much easier to manually edit the terraform states.

[jauntywundrkind]

If you're any good at all at CDK, it's cdk8s is also a very solid clear & clean way to do kubernetes too. https://cdk8s.io/

I'm trying to make the decision for where to go with my home lab, and while Pulumi and Cue look neat, cdk8s seems so predictable & has such clear structure & form to it.

That's said the l1/l2/l3 distinction can be a brute to deal with. There's significant hidden complexity there.

[shepherdjerred]

I use cdk8s for my homelab and absolutely love it. 100% recommend.

Homelab CDKs: https://github.com/shepherdjerred/monorepo/tree/main/package...

Script I wrote to generate types from Helm charts: https://github.com/shepherdjerred/monorepo/tree/main/package...

[juffasdfwa]

I don't see it that way. CDKs, Terraform, Pulumi all have their use cases and trade offs. I use a mix of them.