[jumperjake]

IANAL, but copyright governs redistribution of content not consumption (That's what pirates get busted for). I aslo recall that there was a ruling that footer TOSs aren't enforceable unless the user actively and explicitly agrees to them.

I agree with the GP in that public content is fair game. How do you thing google works?

[bduerst]

Google technically respects robots txt and noindex metatags. OP is arguing the ethics of scraping, not if people are ignoring bot meta tags.

Copyright governs how the content is used, including distribution. The reason people who download videos are not liable is because you have to download the complete content to see the copyright. File sharers have already downloaded the content and are subject to copyright. Bots that scrape can interpret meta tags in the header of the dom, which is why scraping and violating copyright is unethical.