Does anyone know the equivalent of lambda without the 15 mins restriction. Firecracker would fit the bill if it didn't have the bare metal requirements.
The problem with Fargate is the cold startup time and lack of support for GPUs. It's also quite new and there's all sorts of issues with it but hopefully will get resolved.
I think ECS is a better fit if you require GPU or you can do with not scaling to zero. Obviously once Fargate catches up and polishes its kinks it would stand on its own but given the way Cognito is, I have very little faith that things will improve quickly (theres an open ticket in cognito that is over 3 years old).
Cold start would be just as much of a problem on lambda, no? Which is to say, it would or wouldn't be a problem depending on your needs modulo language/runtime choice. If we expect to compute for >15 minutes, cold start is probably not a big deal.
You could use AWS batch. It is like lambda but for more resource intensive uses.
I remember as part of the AWS certification exams this use was explicitly mentioned.
You can trigger it from an s3 upload, for example.
I stood up openfaas for that a few years ago. Set an absurdly high global timeout and something more reasonable for each function. Setup was a twidge finicky (spend a day and rtfm), but I made a couple templates for each language the company used, and people generally had a much easier time deploying via that than anything else we had at the time, even for languages they'd formerly never touched.
I want to like Google Cloud but so many horror stories I've read from people having their production environment shut down and locked out of their accounts because the over zealous fraud detection found an issue or non-chalant account managers.
Not sure why you would need 60 minutes for HTTP invoked functions, that alone should be a signal to offload it to another queue or asynchronous process.
People hear horror stories about Paypal, but it is still massive. I stopped living in fear of everything and decided to adopt stuff that does the job. HN loves to be an echo chamber, especially hating on things. If you only listen to the loud voices, you will end up missing out on things that are actually quite good.
I've used GCP / AppEngine for about a decade now. Built one biz that did $80m gross in its first year. Never even a small threat of being shut down. Recently had a billing issue with GCP and after explaining the problem, they credited me far more than I was even asking for.
I currently have 20k+ servers hitting Cloud Functions (golang) and Cloud SQL 24/7, which is a constant 50 requests/sec... costing me about $100 a month total. It was easy to set up, documentation is well written with clear examples, deployments are all through CI on Github. It just works.
We have a small ETL process (no other ETL in the apps stack so adding something else was silly) that runs on cloud run. It gets a http request to start, puts a lock,status record in the database, forks a worker and returns the id to the http client. The work runs for 10-30 minutes to finish the work and exits and then cloud run shuts us down till the next request.
Like I said we did it this way to avoid adding a proper queue and worker pool to this app so not recommended for heavy traffic but it works
I always find it interesting that GCP employees respond to these sorts of threads with generally helpful comments, but AWS employees rarely respond at all.
Yet at the same time, HN loves to hate on GCP. Keep up the good work @yegle.
Google’s Cloud Run with 60 minutes, reasonable cold starts and even always allocated vCPU which are priced much cheaper until request are received to further help with warm starts.