basically I've written bots for scraping governmental sites, and fashion sites. I've never yet had a problem (although the code on one fashion site looked like all classes were randomly generated with implied some attempt to keep scraping from working), so what I'm wondering is - what kinds of sites actually go through the trouble of tagging you as a bot, since you've got experience care to share?
Ages ago I was one of those bots. Most fun I've had playing cat-and-mouse. Their team was actually quite good. And I learned that Perl can solve any problem.