Y
Hacker News
new
|
ask
|
show
|
jobs
Ask HN: legally can i build search niche web search engine?
6 points
by
umenline
4888 days ago
i want to build search engine that searching dally niche sites , like blogs first page's then present statistics based on the crawling. legally can i do this ?
2 comments
yamural
4888 days ago
There is no legal restriction for crawling any web site. But you should respect robot.txt document for every single web site. Such as:
https://www.facebook.com/robots.txt
http://web.archive.org/robots.txt
link
manidoraisamy
4888 days ago
Yes. You can. Google successfully defended it in the past :
http://en.wikipedia.org/wiki/Field_v._Google
link