Hacker News new | ask | show | jobs
by tnolet 2756 days ago
I run a ton of Puppeteer jobs (300k in the last month), currently on EC2 and Digital Ocean VM's, mostly due to the subtle difficulties of running Puppeteer on Lambda.

Will certainly have a look at this project and contribute where possible.

My main concerns are not so much cold start time, as for my use case this is not really a huge issue, but mainly the performance of Chrome on AWS Lambda boxes. The rendering, navigation etc. needs to be snappy.
2 comments
thesandlord 2755 days ago
Google App Engine and Google Cloud Functions got native support for Puppeteer a few months ago as well. Let me know what you think if you try it out.

https://news.ycombinator.com/item?id=17795626

(I work for Google Cloud)

link
alixaxel 2755 days ago
The performance of puppeteer is super bad on CGF (you can read more about it here https://github.com/GoogleChrome/puppeteer/issues/3120). It would actually be great to have someone really improve this situation instead of dismissing it as a weird IO problem.
link
thesandlord 2755 days ago
Did some research internally, this is being tracked but still no root cause AFAIK :(
link
tnolet 2755 days ago
Would love to use GCF but the performance is terrible (as mentioned) and I need more geographic locations than GCF offers.
link
alixaxel 2755 days ago
I also run hundreds of thousands of puppeteer sessions every month, all on Lambda and so far I'm pretty happy with it, from scalability itself to session performance.

Granted, there are some issues with rendering (fonts, emojis and whatnot) but meanwhile there are solutions available that could be explored.

Feel free to try it out and share your specific challenges on GitHub, I'll do my best to come up with solutions for them.

link
russian_bot 2755 days ago
hi,

out of curiosity, what is it that you do that demands so many sessions? Just webscraping?

link