[kidintech]

Seconded. I tried to do this many years ago for my dissertation and failed, but this would be a dream of mine.

[robertlagrant]

Would it not be possible to create a search engine that only crawls certain sites?

[kidintech]

I was most interested in the offline aspect of it, which I wouldn't know where to even start with if I were to fork.

How do you parse and efficiently store large, unstructured information for arbitrary, unstructured queries?

[stavros]

You put it in a search server, like ElasticSearch or Meili.