Hacker News new | ask | show | jobs
AI scraperbots keep hammering us from residential proxies (fosstodon.org)
5 points by aendruk 2 days ago
1 comments

Been seeing this as well. This rule in Cloudflare gets almost all of them and doesn't seem to hit legitimate traffic. The bot is using old versions of browsers in the agent string. We set this with a non interactive challenge and it's been a real life saver:

(http.user_agent contains "Chrome/146." or http.user_agent contains "Chrome/133." or http.user_agent contains "Chrome/145." or http.user_agent contains "Firefox/135." or http.user_agent contains "Firefox/133." or http.user_agent contains "Chrome/131." or http.user_agent contains "Chrome/116." or http.user_agent contains "Chrome/124." or http.user_agent contains "Chrome/107." or http.user_agent contains "Chrome/104." or http.user_agent contains "Chrome/110." or http.user_agent contains "Chrome/99." or http.user_agent contains "Chrome/119." or http.user_agent contains "Chrome/136." or http.user_agent contains "Chrome/123." or http.user_agent contains "Chrome/100." or http.user_agent contains "Chrome/101." or http.user_agent contains "Chrome/65." or http.user_agent contains "Chrome/108." or http.user_agent contains "Chrome/112." or http.user_agent contains "Chrome/109." or http.user_agent contains "Chrome/103." or http.user_agent contains "Chrome/106." or http.user_agent contains "Chrome/111." or http.user_agent contains "Chrome/105." or http.user_agent contains "Chrome/117." or http.user_agent contains "Firefox/115." or http.user_agent contains "Chrome/135.") and http.referer eq "" and (not cf.client.bot) and (not http.cookie contains "xf_user")

Edit: Note, this also only happens to people who don't have a Xenforo key set, you'll need to remove that or change it to whatever cookie your software sets.