Hacker News new | ask | show | jobs
by josefresco 5806 days ago
"Why aren't sites that scrape content blacklisted?"

What like Google News?

It's a lame joke but it shows you how fine the line is between 'scraping' and 'aggregating' content.

1 comments

Google News doesn't show up in search results the way, e.g., Mahalo might. The only search results I've seen that incorporate Google News are built right into the main results page; I don't click through expecting content and get a Google News page instead.

In fact, I haven't seen any Google-owned scraping or aggregating page in a result that I've clicked through. They are big believers in the theory that you should look at exactly one search results page, not a page that takes you to a page that takes you to (...) the result you actually wanted.

I haven't seen any Google-owned scraping or aggregating page in a result that I've clicked through.

What about Google Health results? Try [Whooping Cough] or similar. Top 'result' is a Google health page whose main column is all content republished from Medline. Right column is essentially 'more results' from News and Scholar.

It's not quite as bad as other paste-together pages of text and more results, but they're creeping in that direction.