[efreak]

Some services ignore global disallow, but will respect rules explicitly targeted at them.

[boxed]

I've put in a hard block for all crawlers on all pages. Works for my scenario I think. Hopefully they don't lie in their user agent. Then it's going to be really bad.