Hacker News new | ask | show | jobs
by wilg 699 days ago
You could maybe still only follow robots.txt rules for Googlebot.