[toast0]

> Since Google uses multiple IP addresses to crawl it’s also difficult to block these type of GET flood

It wouldn't be too hard to block by User-Agent: Mozilla/5.0 (compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html); if you notice the traffic.

Feedfetcher does not fetch robots.txt though; so you'd have to do something in your server config.

[edit: fixed a typo, and agree with the update]

[chr13]

Thanks, I've updated the post.