|
|
|
|
|
by diggan
622 days ago
|
|
> Does any of these scrapers uniquely and unambiguously identify themselves as a bot? It seems like all of them do, yeah: https://github.com/eob/isai/blob/b9060db7dc1a7789b322b8c2838... Not sure if they're really "scrapers" though, if they're initiated by a user for a single webpage/website, more like "user-agents" in that case, unless it automatically fans out from there to get more content. |
|