Hacker News new | ask | show | jobs
by jmnicolas 2095 days ago
I use it to get rendered web pages I want to scrap.
2 comments

I was doing the same for an article reader / archiver I built but it was slow. Did you ever figure out how to speed it up? Cancelling image and media requests helped but it increased the likelihood of being flagged as a bot.
Sorry I never looked at the perf aspect of it since my needs were modest (less than 50 web pages per day). The first thing that would come to my mind is to parallelize it but then it's going to become pretty complex.
*scrape