Hacker News new | ask | show | jobs
by chriskinsman 2389 days ago
Headless chrome/puppeteer...
1 comments

No-no Headless chrome / firefox doesn't work here, since I need to run Selenium on the server and that's the most obvious overhead because any browser, even in a headless mode consumes RAM. This becomes even more critical if you need to process more than 100 images per minute :|
From experience 100 images / min with chrome and puppeteer (you don't need Selenium) is more than doable with less than 1gb ram.
You could roll your own fairly easily with qt webkit and a mere smidgen of c++. But it'd probably be similar yo puppeteer. Maybe you can save some time by not creating a new instance of puppeteer for each image.
Selenium can also snapshot pages.