Hacker News new | ask | show | jobs
by VladVladikoff 3 days ago
In my experience, not really. Each asset is made by a new proxy, eg some TV somewhere, they send every new request via a new IP address, which is a new machine, and it doesn’t matter how long the response takes, they have already rotated to the next ip immediately and sent another request. Most of these providers have millions of IPs, and it generally doesn’t take millions of requests to scrape a website (unless it’s really big!)
1 comments

And each new IP triggers a new Anubis challenge. Isn't it great?
So, once the challenges are solved by the residential proxies (with optimized native implementations [1]). Some unsuspecting TV owner is paying with their electricity bill (not the companies doing the scraping) and we all suffer through challenges.

Yes, great outcome! </s>

[1] https://lock.cmpxchg8b.com/anubis.html

Proxy networks just proxy. They don't let their client run compute on the proxy. That isn't even a possibility in socks5 protocol.