Hacker News new | ask | show | jobs
by 0123456789ABCDE 31 days ago
this reminds: webcrawlers are supposed to include an info url with their user agent header

ex: Googlebot/2.1 (+http://www.google.com/bot.html)