Hacker News new | ask | show | jobs
by lapnitnelav 2932 days ago
There are services dedicated to scrapping which can take care of proxy-ing your requests so you don't have to worry about IP bans.

For example, Scrapinghub's Crawlera (the guys behind the Scrapy python lib)