Hacker News new | ask | show | jobs
by jitl 496 days ago
The terraform documentation explicitly advises you NOT to do this (https://developer.hashicorp.com/terraform/cli/workspaces#whe...)

> In particular, organizations commonly want to create a strong separation between multiple deployments of the same infrastructure serving different development stages or different internal teams. In this case, the backend for each deployment often has different credentials and access controls. CLI workspaces within a working directory use the same backend, so they are not a suitable isolation mechanism for this scenario.

For a practical scenario, you will often need different environments to roll out changes at different times, or to have other slight variance. If you rely solely on variables to be the only difference between environments, you will need a lot of tricky shenanigans to say, create a new dynamodb for a proof of concept only in “dev” but not in prod. Sure, you can use `count = var.env == “dev” ? 1 : 0` but this gets old fast.

Much better to make modules for common stuff, and then compose them in your different environments. Depending on the complexity of your needs, keeping good organization & practice around using modules can be a bit challenging, but it will definitely scale through composition.

Modules also important to make multiple copies of things within an environment, for example to have a cluster in us-west-2 and a cluster in eu-central-1, both are in production environment. I would assume if I started with workspaces I would rapidly hit a point where I want to use it as a module and then need to re-organize things. If you chose workspaces as soon as you want a second region you need a big refactoring, but if you are using modules, you just instantiate the module a second time in your second region.

2 comments

I think all your points are valid, but I've also had good results using workspaces for environments. Here's generally how I structure my terraform primarily targeting AWS.

- 1 Terraform workspace per environment (dev, test, prod, etc.).

- Managing changes to workspaces / environments is done with whatever approach you use for everything else (releasing master using some kind of CICD pipeline or release branches). The Terraform is preferably in the same git repositories as your code but can be separate.

- The CICD tool injects an environment variable into the build to select the appropriate workspace and somehow supplies credentials granting access to a role that can be assumed in the appropriate account.

- A region module / folder that defines the resources you want in each region. This is your "main" module that specifies everything you want.

- Minimal top level terraform that instantiates multiple AWS providers (one for each region) and uses them to create region modules. Any cross region or global resources are also defined here.

- The region module uses submodules to create the actual resources (RDS, VPCs, etc.) as needed.

This approach assumes you want to deploy to all your regions in one go. That may not be the case.

I'm not sure that's completely what they are saying. Under the use cases section, they do acknowledge that workspaces can be used for test environments. I think what they are saying is to align workspaces to your architecture and team structure. If you have the same team managing their components across environments, workspaces could be a fine way of doing it.

The problem with manually composing your environments is that you can lose parity across them. Over time it can be hard to point to what is the source of truth for your architecture. Like, this environment invokes these modules, but this other environment does not! Which is the correct one? But when systems cross team boundaries then they will diverge by design, using workspaces for such cases may not be a good idea.