Hacker News new | ask | show | jobs
by mellosouls 405 days ago
From the repo (clipped):

When using Scraperr, please remember to:

Respect robots.txt

Terms of Service

Rate Limiting

Kudos for promoting ethical usage; makes a change from some of the grifters selling borderline ddos-bots as crawlers.