[LarryMade2]

I do resource directories so have many bits of useful information, a big target for search engines.

Most of my filtering is just to keep the bots from clicking on some forms and excluding them from visitor logs. No problem with most crawlers. The bad ones are the ones that cycle through IPs, mimic agents, and space out their hits, they are the ones that clog my logs - some hit the same page multiple time (I assume some SEO tactic, which is not how my sites rank items).

My best bet might be IP filtering (restricting to US only) - wondering if any have implemented that (is that a big hit on page load time?), I would think with VPNs that doesn't always work that well...

[leros]

I've banned China and Singapore traffic, then seen that traffic pop up from various countries globally. I assume they detect the block and move to rotating proxies.