Hacker News new | ask | show | jobs
by heavenlyblue 2944 days ago
This is what Common Crawl does: http://commoncrawl.org/. I think more people should know about it.