Hacker News new | ask | show | jobs
by palmfacehn 295 days ago
Pick an obscure UA substring like MSIE 3.0 or HP-UX. Preemptively 403 these User Agents, (you'll create your own list). Later in the week you can circle back and distill these 403s down to problematic ASNs. Whack moles as necessary.
1 comments

I've tracked bots that were stuck in a loop no legitimate user would ever get stuck in (basically by circularly following links long past the point of any results). I also decided to filter out what were bots for sure, and it was over a million unique IPs.