Hacker News new | ask | show | jobs
by bagacrap 2431 days ago
I think it's pretty common practice. Otherwise search engine web crawlers would be wreaking havoc.
1 comments

No, both your logic and premise are incorrect. To give just one example, rate-limiting is clearly widespread stateful practice applied to GET requests, and it doesn't cause web crawlers to wreak havoc on anything.