Y
Hacker News
new
|
ask
|
show
|
jobs
by
abolishme
4660 days ago
If you refer to the "how" page, you'll see that they use noindex and nofollow, a robots.txt to discourage crawling, and a blank 403 to any crawler ignoring the robot.txt
http://www.donotlink.com/dnl-how