[maineldc]

We use typescript + pulumi for this. It's pretty amazing. And Pulumi uses Terraform modules under the hood so you get the full power of Terraform with the goodness of Typescript.

Even self hosting your state management in a bucket is simpler with Pulumi since it uses lock files on S3 versus a separate DyanamoDB + S3 combo.

I have been using it in production for 4-5 years and used Terraform for several years before that.

[fishnchips]

> since it uses lock files on S3 versus a separate DyanamoDB + S3 combo

This is disturbing because S3 does not give you guarantees required to implement real locking.

[erik_seaberg]

https://aws.amazon.com/blogs/aws/amazon-s3-update-strong-rea... guarantees that a client's lockfile can always be seen by other clients immediately (which didn't used to be true). If every client backs off and retries after a race, is that enough?

[fishnchips]

I think not, actually. There would still be cases where a race is not detected. I can think of the following sequence: A checks - no lock, B checks - no lock, A writes - success, A reads - match, success, B writes - success, B reads - match, success. A and B both think they now hold the lock.

For locking to work properly you'd need to have a conditional write that would fail if some prerequisite was not met. GCP offers that operation, S3 AFAIK does not.

[erik_seaberg]

I'm no expert but from a quick glance at https://www.pulumi.com/docs/concepts/state/#using-a-self-man... it looks like this might work:

  client A lists s3://bucket/prefix/.pulumi/locks/, sees nothing

  client B lists s3://bucket/prefix/.pulumi/locks/, sees nothing

  client A creates s3://bucket/prefix/.pulumi/locks/unique1.json

  client A lists s3://bucket/prefix/.pulumi/locks/, only sees unique1.json, and proceeds

  client B creates s3://bucket/prefix/.pulumi/locks/unique2.json

  client B lists s3://bucket/prefix/.pulumi/locks/ and sees both unique1.json and unique2.json

  client B assumes it lost a race, deletes s3://bucket/prefix/.pulumi/locks/unique2.json, and retries

There's another mode where both clients pessimistically retry, but fuzzing a retry delay could eventually choose a winner randomly.

[fishnchips]

In this case you have the opposite issue, with no-one actually guaranteed to get a lock even though nothing is holding one. Fuzzed retries may work in practice but theoretically speaking this is a flawed algorithm.

[erik_seaberg]

Hm, I can sort of imagine a way to use lockfile names to claim a random position in a queue of pending changes, but I don't know if anyone has been worried enough to do that. In practice Pulumi seems to give up instead of retrying: https://github.com/pulumi/docs/issues/11679