|
|
|
|
|
by gitgud
2539 days ago
|
|
Crawling is not entirely synonymous with scraping. Crawling web pages is usually done with the specific purpose of search and categorisation, whereas scraping is much more generalized... Data mining, mirroring, avoiding the official API limits etc... Google may have written agreements with Craigslist, they're both enormous companies... Finally, as others have said it's a legal grey area. It's not completely clear and it basically depends on what websites you're scraping, how you use the data and why... Maybe it's best to just ask the website? |
|