Hacker News new | ask | show | jobs
by nirui 1087 days ago
Unless the captcha is annoying enough to a significant degrees, I doubt that it would work. With all the money in the bucket, scrapers can just hire a captcha farm to get pass the captcha with help from a real human.

Also a side note: distributed Web crawler is not unheard of these days, as well as residential IP proxies. Meaning the effectiveness of Proof of Work model maybe also limited.
1 comments
yuuuuuuuu 1087 days ago
How do residential proxies help? Scraping would effectively be bitcoin mining, which costs resources without shortcut.
link
nirui 1086 days ago
Many online services (including Twitter) do employ some kind of IP address scoring system as part of their anti-scraping effort.

These systems tend to treat residential proxies as normal users, and puts less restrictions on them. On the other hand, if the IP address belongs to some (untrusted) IDCs, then the system will enable more annoying restrictions (say rate limits etc) against it, making scraping less efficient.

link