|
|
|
|
|
|
by sp332
4691 days ago
|
|
|
I dont' think that's true. https://news.ycombinator.com/robots.txt User-Agent: *
Disallow: /x?
Disallow: /vote?
Disallow: /reply?
Disallow: /submitted?
Disallow: /submitlink?
Disallow: /threads?
Crawl-delay: 30
Links in comments are marked nofollow but submissions are not. |
|
|