Hacker News new | ask | show | jobs
by jefftk 2501 days ago
The blog post you link has a bunch of alternatives, but I agree they're not great. If there are a lot of webmasters who want to be able to noindex through robots.txt then making the case for adding noindex to the standard would be a good next step.

(Still speaking only for myself)

1 comments

Googlebot actually used to support a noindex rule in robots.txt, but they are removing it.

https://webmasters.googleblog.com/2019/07/a-note-on-unsuppor...

Yes, that was linked above. It looks like this is part of reducing support to what's in the spec?
Oops, yep. I didn't see that context.