[lolc]

This, to me, is the strongest argument to offer these slop generators. It provides an incentive to follow the robots.txt.

[xyzal]

Exactly. You disobey robots file => we'll make your crawl gain a net negative.