|
|
|
|
|
|
by lolinder
634 days ago
|
|
|
But here we're talking about Common Crawl being included in this scheme, which is explicitly designed to make it easier to use them than to make your own bad robot. You block Common Crawl and all you'll be left with is the abusive bots that find workarounds. |
|
|