Hacker News new | ask | show | jobs
by paddy_m 2763 days ago
How did you get the zip down to 50MB. I was under the impression that pandas+numpy was closer to 300MB and bumped up against AWS size limits. I was considering building some hacked together thing with S3

I came to this thread specifically to find out about numpy and pandas on lambda.
2 comments
richstoner 2763 days ago
We've been running a stripped down version of numpy + scipy + matplotlib in lambda. We'd build the environment in a docker container with Amazon linux, manually remove unneeded shared objects and then rezip the local environment before uploading to s3.

A similar method is described here: https://serverlesscode.com/post/deploy-scikitlearn-on-lamba/

Layers should make this entire process easier.

link
dacm 2762 days ago
When I worked on this I used this article as a reference: https://serverless.com/blog/serverless-python-packaging/ and also ended up with a huge image. What that article didn't mention is that the virtual environment folder should be excluded in the serverless config file, as the runtime is provided by boto3. So adding:

package: exclude: - venv/

would reduce the size considerably (to 50 MB in my case)

link