Hacker News new | ask | show | jobs
by landtco 4265 days ago
In theory, you can disallow crawlers and bots via robots.txt. It's not a hard defense per se because crawlers can ignore it.