Hacker News new | ask | show | jobs
by karn97 438 days ago
Why not just rate limit every user to realistic human rates. You just punish anyone behaving like a bot.
2 comments
mrweasel 438 days ago
Because, as pointed out in another post about the same problem: Many of these scrappers make one or two requests from one IP and then move on.
link
guerrilla 438 days ago
Sold. Pay by page retrieval rate.
link