|
|
|
|
|
|
by owlmanatt
4431 days ago
|
|
|
> ii) Should search engines pay the scraped sites if they are charging to access their indexed data? probably some of the scraped sites has a specific license forbidding the search engine to sell their information in any way. Any reputable search engine will respect robots.txt. |
|
|