Y
Hacker News
new
|
ask
|
show
|
jobs
by
freddydumont
415 days ago
While I get your point about Anubis excluding some users, its purpose is to protect the "commons" from those who would abuse and destroy them: big tech crawlers that do not respect robots.txt files.