Y
Hacker News
new
|
ask
|
show
|
jobs
by
jprete
531 days ago
I hope this is working out for you; the original article indicates that at least some of these crawlers move to innocuous user agent strings and change IPs if they get blocked or rate-limited.