[IncreasePosts]

I disagree - this is not a crawler that just blindly stumbles around any random website that it finds. It is more akin to a user agent. The only requests it makes are derived from specific instructions by the user to do so.

Having said that, people may use it as a crawler, just like you might be able to script Firefox to be a crawler, but it is not in itself a crawler.

[dumbfounder]

It doesn't need to be blind stumbling around the web. But you might be right about only grabbing one page, and if you are then I agree that abiding by robots.txt is only going to upset a tiny minority. When they talk about websites it makes me think they are crawling to see all the pages linked to the homepage, because the asking questions part is extremely limited if all it does is look at one page. If they crawl, then I think they need to abide. If they don't, I think it's ok.