[lapser]

I find it really wild that anyone would ever recommend ECS. A developer deploying a service involves:

- Setting up certs (managed as TF) - Setting up ALBs (managed as TF) - Setting up the actual service definition (often done as a JSON, that is passed into TF)

Possibly other things I'm forgetting.

Some other things. It requires a *developer* to know about certs and ALBs and whatever else.

With EKS, this can all be automated. The devops engineer can set it up so that deploying a service automatically sets up certs, LBs etc. Why are we removing such good abstractions for a proprietary system that is *supposed* to be less management overheads, when in reality, it causes devs to do so much more, and understand so much more?

[tedivm]

I honestly don't understand where you're coming from. If a devops engineer can set things up on eks for people to launch without thinking of those things, what's stopping that same engineer from doing similar for ecs?

When I was at Rad AI we went with ECS. I made a terraform module that handled literally everything you're talking about, and developers were able to use that to launch to ECS without even having to think about it. Developers literally launched things in minutes after that, and they didn't have to think about any of those underlying resources.

[jpgvm]

Handing Terraform to developers has it's own host of issues.

A major benefit of k8s that is usually massively overlooked is it's RBAC system and specifically how nice a namespace per team or per service model can be.

It's probably not something a lot of people think about until they need to handle compliance and controls for SOC II and friends but as someone that has done many such audits it's always been great to be able to simply show exactly how can do what on which service in which environment in a completely declarative way.

You can try achieve the same things with AWS IAM but the sheer complexity of it makes it hard to sell to auditors which have come to associate "Terraform == god powers" and convincing them that you have locked it down enough to safely hand to app teams is... tiresome.

[alien_]

The OP here.

What you say may make sense for a large corporation with hundreds of developers from many teams, all sharing a single cluster, but remember this is a pre-revenue startup with a single dev team of less than a dozen people.

But then with a large cluster you will struggle with splitting the costs. In such scenarios I'd rather give each team its own AWS account and have some devops people set up everything from the landing zone.

In this particular case, every service is set up from less than 100 lines of Terraform, which includes Docker image build and push, as well as the task and service definition that deploys that docker image.

yes, they need to handle Terraform, but it's really not so different from the previous Docker-compose YAML file, not to mention the way it would look if converted to K8s YAML.

[beaviskhan]

Make Terraform run only off git repos, and control commit rights to that repo. That's been a successful approach for me in the past when dealing with auditors.

[tedivm]

Huh, I've had no issues with AWS IAM or Auditors when it comes to Terraform. I've managed compliance at multiple startups too.

[alien_]

The OP here, thanks for the comment!

Why does the developer need to care about the certs and ALBs? The devops engineer you need to set up all those controllers could as well deploy those resources from Terraform.

As I showed in the diagrams from the article this application has a single ALB and a single cert per environment and the internal services only talk to each other through the rabbit MQ queue.

DNS, ALB and TLS certs could be easily handled from just a few lines of Terraform, and nobody needs to touch it ever again.

With EKS you would need multiple controllers and multiple annotations controlling them, and then each controller will end up setting up a single resource per environment.

The controllers make sense if you have a ton of distinct applications sharing the same clusters, but this is not the case here, and would be overkill.

[lapser]

> DNS, ALB and TLS certs could be easily handled from just a few lines of Terraform, and nobody needs to touch it ever again.

Welcome to reality, where this is not the case.

I'm currently working at a company where we're using TF and ECS, and app specific infra is supposedly owned by the service developers.

In reality, what happens is devs write up some janky terraform, potentially using the modules we provide, and then when something goes wrong, they come to us cos they accidentally messed around with the state or whatever. DNS records change. ALB listener rules need to change.

[NovemberWhiskey]

That seems a strange way to look at things to me. If you're going to give credit for things that a devops engineer can do inside the Kubernetes platform, why not given equivalent credit for what a devops engineer can do with a Terraform module that would achieve substantially similar levels of automation and integration with ECS?

[moduspol]

Also weird to leave out which things are versioned things that must be installed, maintained, and upgraded by you (e.g. cert-manager, an ALB controller, the Kubernetes control plane) that do not apply to a Terraform (or CloudFormation)-based deployment to ECS.

[honkycat]

Agreed. After recently finishing up a migration off ECS, it is madness and feels like OP just wanted a contrarian take.

Honestly, if they had said: "So instead we set up some bare-metal EC2 instances" I would be on-board.

[alien_]

The OP here, thanks for the comment!

It was definitely not about being contrarian but about offering first and foremost a more cost effective but still relatively simple, scalable and robust alternative to their current setup.

They have a single small team of less than a dozen people, all working on a single application, with a single frontend component.

Imagine instead this team managing a K8s setup with DNS, ALB and SSL controllers that each set up a single resource. I personally find that overkill.