Hacker News new | ask | show | jobs
by alyandon 88 days ago
It seems that AI-scraping hysteria/paranoia has reached a point that more and more resource sites completely block me without any recourse (e.g. having an account) if my traffic is coming from a datacenter IP. I really wish sites blocked agents/IPs based on actual bad behavior instead of reflexively blocking by IP != residential/mobile.

I do understand the frustration though. I've had to deal with bad bots scraping every crevice of a few web based services I host and while most went away after I tweaked robots.txt there were a few I've had to blanket ban IPs assigned to entire ASNs because the bots on their networks refused to play nice.