Hacker News new | ask | show | jobs
by strangattractor 272 days ago
Great Firewall actually. Robots.txt depended on the integrity of the companies crawling. I think they have demonstrated how much integrity they actually have:)