Y
Hacker News
new
|
ask
|
show
|
jobs
by
Ajay-p
848 days ago
Does Reddit really block all search engines, or do "all" search engines abide by No Spiders? If it's freely available it's ripe for scraping, no matter what Reddit may say.