Y
Hacker News
new
|
ask
|
show
|
jobs
by
hoppla
1311 days ago
Another neat trick is to add a link in robots.txt and instruct bots to stay away. If they don’t, you add them to your blocklist
1 comments
justin_oaks
1311 days ago
I was confused at what you were saying at first. For those that may also be confused:
You can add something like this to your robots.txt:
User-agent: * Disallow: /some/unguessible/url
And then you ban any IPs/bots that visit that URL.
link
You can add something like this to your robots.txt:
And then you ban any IPs/bots that visit that URL.