[frmdstryr]

I recently found https://commoncrawl.org/ and started working on my own personal search engine. I'm surprised how easy it is to actually get decent results with just metadata that show up nowhere in Google/DDG.

[closedloop129]

Heads up to https://news.ycombinator.com/item?id=31410042

marginalia_nu, the author of another search engine, is looking for ways to cooperate.

[guerrilla]

What's the license for this? It isn't obvious from the website and I got bored of looking.

[buo]

Not a license, but there are some terms of use: https://commoncrawl.org/terms-of-use/