Hacker News new | ask | show | jobs
by nevi-me 1038 days ago
It could be better to opt those articles out of the crawler. Unless that's more effort. If articles included the year and month in the URL prefix, I would disallow /201* instead.