Hacker News new | ask | show | jobs
by digitallyfree 1516 days ago
Uh, you don't really "allow" them to visit your website. All you can do is disallow them from visiting by blocking them using robots.txt. If you don't have a robots.txt stating otherwise they will crawl and index your site.

UPDATE: I suppose if you specifically allow the Googlebot via robots.txt, then in that case they could probably argue that you gave them permission to access the site.
2 comments
creato 1516 days ago
If you put a website on the internet, you are granting anyone or anything permission to look at it by default. The alternative seems obviously untenable.
link
digitallyfree 1516 days ago
Exactly, and this is why the TOS mentioned in the comment is absurd. Basically that means that if the TOS is legally valid, then technically Google can force any arbitary terms on you if you simply put a website on the internet. This would be the same as saying that all citizens of America are subject to this agreement, whether if they've seen it or not.

This is worse than shrinkwrap agreements and TOS banners at the bottom of a website, since for those you could at least argue that the person who opened the box/visited the site saw the agreement. But in this case, there is absolutely nothing to indicate that the person who created the website even knew that Google as an entity existed.

link
genewitch 1516 days ago
https://www.yahoo.com/news/missouri-governor-says-security-1...

In theory you're right. In practice, just because something responds publicly does not mean it's public.

If someone left the vault door open on an ATM it doesn't mean you can just have the money, or even take pictures of the insides.

Obviously, in IT, people should strive to prevent leaking information, but that's obviously going to be an issue for a great long while.

link
bduerst 1516 days ago
That article is about a tech-illiterate governor using his station to threaten prosecution of the press... for discussing a 10-yr old bug that's already well known publicly.

If anything, that's more to the discussion of giving social security numbers to inept IT departments.

link
Teandw 1516 days ago
When you put a website live on the internet, you're giving everyone permission to access it, including Google, unless you state otherwise. (Or put processes in place to make it non-public.)

So yes, you do allow them to visit your website.

link