Hacker News new | ask | show | jobs
by hoppla 1311 days ago
Another neat trick is to add a link in robots.txt and instruct bots to stay away. If they don’t, you add them to your blocklist
1 comments

I was confused at what you were saying at first. For those that may also be confused:

You can add something like this to your robots.txt:

    User-agent: *
    Disallow: /some/unguessible/url
And then you ban any IPs/bots that visit that URL.