Y
Hacker News
new
|
ask
|
show
|
jobs
by
gcmartinelli
5072 days ago
If everyone could implement their own scrapper they would be able to follow robots.txt guidelines/limits, but HN servers might take a big blow it it gets popular.