Hacker News new | ask | show | jobs
by textfiles 156 days ago
They absolutely are.
1 comments
razakel 156 days ago
Of course they are. Had to block anything at work coming from one certain company because it wasn't respecting robots.txt and the bill was just getting silly.
link