[jumperjake]

You're confusing redistribution with scraping. Scraping publicly-accessible content is legal. Redistributing it is not (at least not in the US).

Google scraps webpages as a core competency.

[chinathrow]

Google also adheres to the robots.txt standard. Most of the scrapers I block don't.

[jumperjake]

Not correct. Google will completely ignore the rules in robots.txt if it deems it acceptable. I think there's a link to this somewhere in this comment page.

[chinathrow]

They do not index the content but might add the URL, correct. You can have a meta noindex present and they won't index even the URL.