[Tehnix]

I really hope this helps with the cold start times on Lambda. We were currently looking heavily into moving our API from Lambda to EKS, but if this impacts cold start times, I think we will look at how it ends up looking like in practice.

[sudhirj]

Most code start time problems on Lambda I've seen are VPC related - public network Lambdas start in milliseconds, with the main lag being the userspace code startup time.

Starting a Lambda inside a VPC involves attaching a high security network adapter individually to each running process, which is likely what takes so long. I assume AWS is working on that, though, they've claimed some speedups unofficially.

If your security model allows, try running your Lambdas off-VPC.

[Tehnix]

The VPC startup times are insane, so we quickly move our lambdas out of that, accepting the trade off.

Our normal cold starts are in the 1-2 second range, and the app initialization comes after. Too high for an API facing users :/

[cddotdotslash]

We got around this with a bit of a hack - use a CloudWatch event to trigger a dummy invocation of your function every five minutes. This keeps the container "hot" and reduces the start time (and is negligible cost-wise). This won't fix the cold starts when the function scales up, but it does reduce latency for 99% of our API requests.

[Tehnix]

We're already doing this, but unfortunately this doesn't work well when you are expecting lots of parallel calls, and various times.

The obvious solution would be to just merge microservices into the same lambda, but then we'd rather switch to EKS or smth, and actually be able to utilize microservice architecture fully.

To give a little more context, we have a bunch of microservices exposing GraphQL endpoints/schemas. We then have a Gateway which stitches these together, and exposes a public GraphQL schema. Because of the flexibility (by design) of GraphQL, we can easily end up invoking multiple parallel calls to several microservices, when the schema gets transformed in the Gateway.

This works really well, and gives a lot of flexibility in designing our APIs, especially utilizing microservices to the full extent. It also works really well when the lambdas are already warm, but when we then get one cold start, amongst them all, suddenly we go from responses in ms, to responses in seconds, which I don't think is acceptable.

We've been shaving off things here and there, but we are at the mercy of cold starts more of less. So our current plan is to migrate to an EKS setup, we just need to get a fully automated deployment story going, to replace our current CI/CD setup, which heavily uses the serverless framework.

[vdfs]

Isn't using a server better in this case? Or Lambda have some benefits in this setup?

[cddotdotslash]

It's still insanely cheap. You could have millions of executions per month and only pay $0.50. But if you needed to, it could scale up to billions of invocation nearly instantly, something a standard server would have trouble doing as easily as Lambda does.

[markonen]

Then again, you are doing the hacks you describe because it is not scaling up nearly instantly. The cold start delays are not only an issue when scaling from zero to one, they hit you whenever you scale the capacity up.

[sudhirj]

That's odd. Are you running the interpreter runtimes (NodeJS/PHP), compiled binaries (Go) or VMs like Java?

[Tehnix]

We are using Node.js. From what I’ve seen online, Python should be the fastest though.

Edit: wanted to add, that from what I’ve gathered from people testing online, bundle size didn’t really matter, but perhaps someone else has some information that points to the contrary?

[sudhirj]

Bundle size is rarely equivalent to size of initial executable code. i.e. I can have a giant photo of my dog in the bundle but it won't necessarily affect the start up time of the node index.js. I think there is some kind of effect in that the bundle must be downloaded to the Lambda container server from S3, but that seems pretty fast, and it's likely cached there for a while.

[mneves]

One solution is invoking a scheduled Lambda (with a test payload) at regular intervals to keep the function warm.