Imagine CF being Terraform + Terraform Cloud (only free!) - but more reliable and having real changesets with more predictable behavior and the state being the true AWS state, not some projection of it within Terraform.
I've had too many CF stacks get stuck in states that don't allow for rolling forward or backwards to buy this. You have to purchase a premium support plan to get access to the AWS personnel who can help you out.
CF does actually kind of try to behave transactionally, which is interesting--you don't often get stuck between states like you do with Terraform, but I rarely have had much of a problem getting Terraform out of one of these states either.
This isn't to speak about the abysmal expressiveness of CloudFormation. Reuse is a joke--I'm tempted to call it a programming language in which you encode the AST manually as YAML/JSON, but that would be overly generous because any programming language AST allows you to express the concept of a function that takes complex arguments (e.g., an object, list, or list of objects).
Further still, if you want to extend CloudFormation to support third party services (e.g., if you want to create a user pool in your identity management provider for your new stack, or otherwise manipulate non-AWS APIs), you have to write and run your own lambdas--which each require their own infra-as-code--which just takes a lot of effort (not to mention how hard it is to write these correctly) and so you end up compromising with a half-manual workflow, missing out on much of the promise of infra-as-code.
I don't love Terraform, mind you. I think there's a Better Way out there somewhere (maybe it's Pulumi or CDK, I'm not sure), but CloudFormation is brutal.
This is... quite the assertion. Terraform has historically been _far_ more reliable than CloudFormation, and perfectly capable of getting itself into unrecoverable states. Furthermore, the behaviour has _never_ been predictable - CF only even grew the ability to preview changes relatively recently.
(Disclaimer: I worked on Terraform at HashiCorp, and on other provisioning tools since).
For a person who uses Terraform daily, I won't agree with you. So many times I had to manually alter state files, and often contribute to basic providers, and wait months to get PRs merged in. It used to be that CloudFormation lagged behind Terraform for AWS, but not anymore. Now, it's quite the opposite and you see people using CloudFormation resource within Terraform as they are stuck into using plain Terraform and yet don't want to manage resource manually.
You won’t find argument from me that HashiCorp have let maintenance slide, but there is still no conceivable way that CFN is more reliable in the general case.
Hmmm, just searched GitHub for aws_cloudformation_stack and found 375 public repositories. I'm sure those references are orders of magnitude more in the private repos. In other words, you still need CloudFormation with Terraform on AWS... so, why use the patchwork then?
The stregth of TF is the community and quality of docs. I use providers to manage an infra that is a mix of AWS + Heroku + external services like Rollbar and that just comes easy with TF. The point of it being paid (although I don't use any of their paid services), is that it is their core business and they are forced to continuously improve on it. TF 5 years ago sucked, but TF in 2021 is super smooth.
The only problem is Cloudformation is strictly AWS resources, so unless you're absolutely 100% sure you'll never need anything outside of AWS ecosystem, I always recommend using TF.
If you're invested, you can use extensions [0] which seem like they're more well-defined. It sounds like this is what APN partners like Datadog use to implement 3rd party resources.
This is true, of course, but you can have a Terraform CloudFormation resource, too, and pass data from other resources back and forth. Since when we don't like using the best tool for the job?!
For companies the likelihood of not needing stuff outside of AWS is relatively low. Unless you're okay with managing other applications by hand, which is probably the wrong attitude.
CF does actually kind of try to behave transactionally, which is interesting--you don't often get stuck between states like you do with Terraform, but I rarely have had much of a problem getting Terraform out of one of these states either.
This isn't to speak about the abysmal expressiveness of CloudFormation. Reuse is a joke--I'm tempted to call it a programming language in which you encode the AST manually as YAML/JSON, but that would be overly generous because any programming language AST allows you to express the concept of a function that takes complex arguments (e.g., an object, list, or list of objects).
Further still, if you want to extend CloudFormation to support third party services (e.g., if you want to create a user pool in your identity management provider for your new stack, or otherwise manipulate non-AWS APIs), you have to write and run your own lambdas--which each require their own infra-as-code--which just takes a lot of effort (not to mention how hard it is to write these correctly) and so you end up compromising with a half-manual workflow, missing out on much of the promise of infra-as-code.
I don't love Terraform, mind you. I think there's a Better Way out there somewhere (maybe it's Pulumi or CDK, I'm not sure), but CloudFormation is brutal.