[manx]

I thought about creating a search engine using https://github.com/phiresky/sql.js-httpvfs, commoncrawl and cloudflare R2. But never found the time to start...

[oersted]

You will like this then, that was the main demo from the Quickwit team.

https://common-crawl.quickwit.io/

[mallets]

Many things seem feasible with competitive object storage pricing. Still needs a little a bit of local caching to reduce read requests and origin abuse.

I think rclone mount can do the same thing with its chunked reads + cache, wonder what's the memory overhead for the process.