|
|
|
|
|
|
by sunshine-o
24 days ago
|
|
|
Thanks you so much this is highly very valuable information. > Thanks to pointing out kiwix. I'll give it a try. I see YaCy works with ZIM files [0] packaged by Kiwix so this is great. In theory if you run YaCy kiwix is not necessary but they do package already valuable sites likes Wikipedia, iFixit, archwiki, etc. [0] so you do not have the worry of your crawler to be blocked and have local copy anyway [1]. So a lot of bandwidth and headache saved. - [0] https://github.com/yacy/yacy_search_server/tree/master/sourc... - [1] https://browse.library.kiwix.org/#lang=eng |
|
|