Hacker News new | ask | show | jobs
by Gigachad 1459 days ago
Anything private should ideally be put behind authentication. If that isn't possible, than robots.txt. Search engines are _meant_ to index everything that is publicly accessible and not blacklisted on robots.txt.