Hacker News new | ask | show | jobs
by muppetman 1351 days ago
I didn't say it was private. It's obviously public, I give you the URL and you can pull it up. I'm not stupid enough to think that robots.txt means no one can read it! There's some RBAC control so that you need to be logged in etc if I publish anything I wish to remain private. Most _reputable_ organisiations will respect robots.txt and not crawl your site, nor cache/archive your data if you request it. What I want is the decision that someone's going to archive/store my website somewhere else to be MY decision, not theirs.