[senko]

If robots.txt were a thing, Google would never have built its search engine. Everyone would just allow Altavista and disallow everyone else, like they do with Googlebot and Bingbot now.

Also, the issue here is not that Google banned their tool from accessing YouTube because the tool didn't obey robots.txt. The issue is that Google, the search engine, nuked their site, because they didn't like its content.