[s_Hogg]

Yeah we have a 10Mb state file and it already takes 30-45 min to deploy. The main way it happens is if someone cancels a CI/CD job because they're realised something is wrong and don't want to wait that long to try again. That's not an issue if there isn't a state file, but yeah I can see how there would be a tradeoff possibly associated.

[Aeolun]

Pulumi is pretty good about keeping track of in-progress state. If you cancel and restart a new job, it’ll first ask you what to do with the in-progress resources.

[thedougd]

IME the plan times explode because the refreshes hit the cloud provider API rate limits.

[dronacharya]

That does seem like quite a large state file. Would you be able to shed some light on the sorts of of resources that you're provisioning? I've found it to be useful to split dependencies between cloud resources, and to add links via data lookups. At any reasonable scale, I've found splitting Terraform state to be crucial.

For example: network resources and Kubernetes clusters are created separately, and networking is a required dependency for Kubernetes clusters. Resources are associated with environments and regions, and modules take in environments and regions as parameters.

In my setup, we have 200+ state files, and the largest is around 80KB.

[rjbwork]

How do you manage the meta DAG between all of that stuff in the event of, e.g. complete disaster recovery?