Hacker News new | ask | show | jobs
by imduffy15 2262 days ago
https://scrapinghub.com/guides/web-scraping-best-practices/ may be of interest to you.