Hacker News new | ask | show | jobs
by notatoad 1917 days ago
if you post information publicly on the internet, google is entitled to scrape it. you've opted in by publishing it.

if you want to specifically exclude one entity from accessing information that you've posted for anybody to see, i'm not sure how there's a way that could be "opt-in"

2 comments

Google is entitled to scrape it, but are they entitled to display the content on their site, the results pages? Everything in the instant answers is content that deserves to be displayed on its creators page, along with whatever monetisation the creator chooses.
You could do this using a robots.txt file (assuming the scraper obeys it, of course).