Hacker News new | ask | show | jobs
by PatchRequest 27 days ago
I manually blacklisted .gov from being crawled on my side. felt like it wasn't worth the potential trouble
1 comments

Meanwhile you crawl anybody else?

.gov isn't magically any more able to take you to court than a private org