Do you feel bad at all about apparently making a business out of crawlers, but still apparently viewing it as bad enough that you want countermeasures against it? Don't you feel a slight bit hypocritical about this?
I don't think I'm being hypocritical. I have no issues if people crawl my site, I even identify who they are and give them access to a private API. I do not generate any income from that website though. I provide a service because I love doing it and I cover all the costs. Bots do increase my cost so I choose to limit their activity. Crawl me, but do so using my rules.
One of the alternatives is charging for my service but bots are not my users problem, they are mine.
One of the alternatives is charging for my service but bots are not my users problem, they are mine.