[waffletower]

The management of infrastructure via Terraform has a hidden engineering cost that should also be considered. Engineers can much more easily maintain, learn and introspect infrastructure via Kubernetes, despite its own complexity, given the immature, inconsistent and undeniably awkward qualities of the Terraform toolchain. Engineering time is expensive -- the morass of Terraform can easily quadruple engineering efforts.

[twalla]

As a something of a k8s maximalist, I kind of disagree here. I think, especially for early stage and smaller teams, TF ends up being "closer to the metal" in the sense that there are fewer concepts and abstractions that need to be understood before an engineer can build a model of the resources they want, how they are grouped and how state reconciliation works. With k8s you're really just trading out crappy third party modules and providers for crappy operators and controllers.

I do think that as organizations grow, the ability for components to be defined in smaller units without being enmeshed in a big-ass tf dependency graph is a big draw of the controller model. The flipside is this comes with accepting the operational overhead of k8s plus the attendant controllers/operators you're running and hiring/staffing accordingly. There are ways you can structure your terraform that avoids creating the tight coupling some folks don't like where you have to literally define the entire universe to change a machine image. Not to mention, there do exist tools that allow you to inspect and visualize tf state.

[pharmakom]

I think Terraform should consider making targeted applies a first class feature/workflow.

Right now, Terraform maximalism requires reproducible builds, which is not something most orgs can achieve.

[KaiserPro]

> Engineers can much more easily maintain, learn and introspect infrastructure via Kubernetes, despite its own complexity.

Citation needed.

K8s has a whole bunch of footguns that people who don't want to manage infra can easily blunder into.

Terraform and ecs is not immature, and its fairly simple to maintain especially if they are just pushing updates without significant infra changes. (ie bumping the container version)

> Engineering time is expensive

which is why ECS is probably better, because its good enough for running a few containers that talk to a load balancer.

[frank_nitti]

ECS will always have the major disadvantage of strongly coupling your infrastructure and often your code to AWS.

They will continue to make it more appealing to lock your software into their platform than to go with their thinner facilities for OSS, doing the minimum to keep up to date with trends in open source, just enough to lure you in and create “easier” paths until you can’t afford to leave.

We have this problem with Azure - sure it’s easier to get a knucklehead to push buttons and get an app running, but after years you’ll be scrambling to reduce costs. Good luck with that when all of your terraforms use Azure Resource Manager and all of your source code uses Azure Functions. Being stuck with microsoft/amazon and a team of engineers who spent their time learning vendor-specific skills instead of the open source tech that enables it, sounds awful.

[KaiserPro]

Look, the issue is this for a small business: you have 5 engineers, do you spend $150 a month to pay AWS to look after hosting your stuff? or do you pay an engineer >>$150 a month to create, manage and maintain a bespoke infrastructure?

Sure, you might be lucky with the engineers you have, they might be savvy enough to wrangle a couple of hosts for you. But are they backed up? what's the disaster recovery procedure like? How do you rotate keys/passcodes, how do you audit who has access?

Unless you are doing something wrong, your biggest costs are engineering time.

> disadvantage of strongly coupling your infrastructure and often your code to AWS.

you can say the same thing about any infrastructure. yes, you can migrate k8s from one physical host to another. But, for that to be effective, you need to not be using any manage services. So that means you're on the hook for all the painful things like DB state and recovery, messaging systems, etc,etc,etc.

Then you think as a business, what are you actually spending money to do? maintain the code that makes you money, or maintain the system underneath it, just in case you might need to move to save opex thats almost certainly going to be less than one engineer.

98% of companies have no issue with scale. They have issues with availability, features, backups, speed. exchanged 10% of an engineer's salary to never really have to deal with any of those issues is a good deal for most[1] companies

[1] most, but not all. However unless you are getting close to spending 1 engineer in AWS fees, moving to self hosting is nothing but premature optimisation.

[lijok]

> Engineers can much more easily maintain, learn and introspect infrastructure via Kubernetes

Hahahaha

[LostLocalMan]

I personally find using CDK over terraform to actually have a performance multiplier rather than a cost. So much so that I end up using CDK8s to manage my Kubernetes resources as well