Hacker News new | ask | show | jobs
by diggan 622 days ago
> Does any of these scrapers uniquely and unambiguously identify themselves as a bot?

It seems like all of them do, yeah: https://github.com/eob/isai/blob/b9060db7dc1a7789b322b8c2838...

Not sure if they're really "scrapers" though, if they're initiated by a user for a single webpage/website, more like "user-agents" in that case, unless it automatically fans out from there to get more content.