[elmerfud]

Is the crawling not respecting your robots file? You don't state a reason for wanting to block the behavior other than it was fun to build the tooling to discover it for you.

Personally I don't see a reason to block non abusive things from crawling a site. Sites are there to be found and read. Indexers, archivers, etc... are normal things that may provide a non obvious benefit.

This seems to be a judgement call on if you believe the the actions are nefarious or not.