[amirouche]

Did you think about database dump of popular services like HN, SO or Wikipedia to speed up crawling and revelance?

[deusu]

Yes. I have downloaded several data dumps, but haven't gotten around to import them yet.