|
|
|
|
|
|
by drbscl
174 days ago
|
|
|
> So fuzzycanary also checks user agents I wouldn't be so surprised if they often fake user agents to be honest. Sure, it 'll stop the "more honest" ones (but then, actual honest scrapers would respect robots.txt) Cool idea though! |
|
|