Y
Hacker News
new
|
ask
|
show
|
jobs
by
saghm
19 days ago
Couldn't they just rate limit them? Are they literally using a new IP for every request?
2 comments
anematode
19 days ago
There are indeed scrapers which use tens of thousands of distinct IPs, and so rate limiting them isn't a solution.
link
sroussey
19 days ago
We used to call this snow shoeing.
link
kstrauser
19 days ago
In my experience, yes. At the peak of a scraper flood I was dealing with, I'd say about 90% of the traffic was from a unique IP. I'd never seen anything quite like that before.
link