[cjk2]

Killer one that has shafted us is tying everything into popular deployment and CI tooling. Your product should be buildable, deployable and tested on a standalone workstation at any time. We have lost literally days fucking around with broken products which get their tentacles into that shit.

I'd use AWS + ECR + ECS personally. Stay away from kubernetes until you have a fairly large deployment as it requires huge administrative overhead to keep it up to date and well managed and a lot of knowledge overhead. Keep your containers entirely portable. Make sure you understand persistent storage and backups for any state.

Also stay the hell away from anything which is not portable between cloud vendors.

As for observability, carefully price up buy vs build because buy gets extremely expensive. Last thing you want is a $500k bill for logging one day.

And importantly every time you do something in any major public cloud, someone is making money out of you and it affects your bottom line. You need to develop an accounting mentality. Nothing is free. Build around that assumption. Favour simple, scalable architectures, not complex sprawling microservices.

[fatnoah]

> Killer one that has shafted us is tying everything into popular deployment and CI tooling. Your product should be buildable, deployable and tested on a standalone workstation at any time

I've currently a VP Eng at a "growth stage" startup, and the state of things when I joined was that about 75% of a dev's time was spent fighting with the external services and data. Someone would spend an entire sprint on a feature, but most of the time was actually fighting with our IdP.

A huge effort and focus (that I had to beat into everyone's head) was that being able to a) run everything locally and/or b) have reasonable fakes for external dependencies means that we can spend far more time building actual product stuff and far less time debugging infra issues.

[sanderjd]

> Also stay the hell away from anything which is not portable between cloud vendors.

Doesn't ECS violate this? This is something I've preferred in theory about kubernetes. At least in theory, it's supported everywhere. But I have also wondered whether it would be just as "easy" in practice to move from ECS to another cloud, as it is with EKS. (That is, neither would actually be easy.)

[cjk2]

The metadata and configuration does violate this but the containers are portable if you care enough to make sure that they don't depend on anything AWS specific.

[sanderjd]

Gotcha!

[infecto]

ECS is the way imo. I dislike the complexity in auth around cloud services but ECS is a really nice way to build scalable systems cheaply. I am a big fan of using the underlying Fargate spot machines within ECS.

[verelo]

I inherited a project on fly.io but before that I’ve used ECS as my primary platform. While ECS is kubernetes lite, fly is ECS but a little more dev friendly in the realm of not dealing with the complexities of IAM etc. It has other downfalls, but it’s the sweet spot for my early stage products right low…although i do expect I’ll end up on ECS if this is going well a year from now.

[djboz]

If the rest of your infra is on AWS(e.g., S3/RDS/Dynamo), why not just start with ECS? Egress fee's would seem to take a toll after a bit using Fly.io

[verelo]

Mostly just because it’s there already, and after getting familiar with it i really like how fast it is to get a project started. Ecs is great, but fly is 10x easier to deploy to.

[tootie]

I love ECS. But I'd also tip my cap to Azure App Service. They're both full flexible models that utilize the core services of public cloud but limit your exposure to the bare necessities.

CDN->Load Balancer->autoscale docker->database. That serves 90% of use cases but you'll have access to the myriad other services they offer for anything else that comes up.

[icedchai]

On GCP, Cloud Run is also a nice solution with minimal complexity. It's very similar to Fargate / ECS.

[zzyzxd]

> I'd use AWS + ECR + ECS personally. Stay away from kubernetes until you have a fairly large deployment as it requires huge administrative overhead to keep it up to date and well managed and a lot of knowledge overhead.

> ...

> Also stay the hell away from anything which is not portable between cloud vendors.

But if your team happens to have Kubernetes expertise, I would say go for Kubernetes.

Anything after building container images will be somewhat cloud-specific regardless. And arguably it can be easier to hire infra/platform folks with Kubernetes skillsets later when you need to.

[fourfour3]

ECS is terribly underrated, I agree.

Most people don’t need kubernetes, and ECS + Terraform is very approachable and easy to get started with, especially with Fargate for compute.