Hacker News new | ask | show | jobs
by artpar 3212 days ago
I just completed building this:

https://awesomelist-fi.nder.site

Taking jnv/lists[1] as root url, crawling every awesome list on github, and project front pages. Indexing all the text using bleve search library [2]. Serving cold with semantic-ui.

The crawler is a single threaded slow crawler, that keeps on crawling, starts at the head again after completing the tree.

[1] https://github.com/jnv/lists [2] http://www.blevesearch.com/

edit: https