Hacker News new | ask | show | jobs
by Jonhvmp 116 days ago
Yeah, those selective hits scream custom scrapers or AI data hunters. To track 'em:

- Parse logs: zcat access.log.* | awk '{print $1,$7}' | sort | uniq -c | sort -nr | head -20

Shows top IPs/paths. Whois suspicious ones.

- Add JS fingerprinting (canvas hashing, WebGL) to log real vs headless.

- Bait pages with unique content.

Set up alerts on anomalies. Caught some sneaky ones that way!