Hacker News new | ask | show | jobs
by oh_no 37 days ago
except google does respect robots.txt so you do have a choice?
1 comments
PunchyHamster 37 days ago
still respects robots.txt
link