Hacker News new | ask | show | jobs
by Bender 347 days ago
For my silly hobby sites I just return status 444 close the connection for anything that has case-insentive "bot" in the UA requesting anything other than robots.txt, humans.txt, favicon.ico, etc... This would also drop search engines but I blackhole route most of their CIDR blocks. I'm probably the only one here that would do this.
1 comments

How does a bot scraping your silly hobby sites for any purpose harm or negatively affect you in any way?
Depends if they hit a site enough to make it cost something. It's not hard for bots to flood servers.
Only if they push me over my bandwidth limits but they can't do that if I just drop them on the floor.