Hacker News new | ask | show | jobs
by bo1024 698 days ago
They say this:

> We have IP-based rate limiting in place for many of our endpoints, however these crawlers are coming from a large number of IP addresses, so our rate limiting is not effective.

Do you have something else in mind? Just shut down the whole site after a certain limit?