|
|
|
|
|
|
by Onavo
410 days ago
|
|
|
In the past month there were dozens of posts about using proof of work and other methods to defeat crawlers. I don't think most websites tolerate heavy crawling in the era of Vercel/AWS's serverless "per request" and bandwidth billing. |
|
|