|
|
|
|
|
by alyandon
88 days ago
|
|
It seems that AI-scraping hysteria/paranoia has reached a point that more and more resource sites completely block me without any recourse (e.g. having an account) if my traffic is coming from a datacenter IP. I really wish sites blocked agents/IPs based on actual bad behavior instead of reflexively blocking by IP != residential/mobile. I do understand the frustration though. I've had to deal with bad bots scraping every crevice of a few web based services I host and while most went away after I tweaked robots.txt there were a few I've had to blanket ban IPs assigned to entire ASNs because the bots on their networks refused to play nice. |
|