Hacker News new | ask | show | jobs
by slang800 3526 days ago
Yeah - there's a ton of immutable URLs on the web. All of CDNJS/JSDelivr/Google Hosted Libraries, most of raw.github.com, all Imgur & Instagram images, all YouTube video streams (excluding annotations & subtitles), and all torrent file caching sites (like itorrents.org). There are probably some large ones I'm forgetting about, but just mapping immutable URLs to IPFS could probably cover 1/3rd of Internet traffic.

Check out https://github.com/ipfs/archives to learn more.
1 comments
clueless404 3526 days ago
IPFS archives look like something entirely different from what the grandparent was talking about.

Sure, you can manually publish archives over IPFS, but that's not something that automatically creates an IPFS cache copy of whatever you are surfing.

link
slang800 3523 days ago
IPFS archives is the effort that's going on right now to archive sites. Eventually there will be a system for automatically scraping & re-publishing content on IPFS.
link
clueless404 3523 days ago
Fair enough. Eventually somebody will have to do a lot of work to get all that done then.
link
slang800 3522 days ago
Right now storage space and inefficiencies in the reference IPFS implementation are the biggest problems I've hit. Downloading sites is easy enough with grab-site, but my 24TB storage server is getting pretty full :( ... Gotta get more disks.
link
clueless404 3522 days ago
Say you grab a site. How do you announce that fact, verify that it is an unmodified copy, sync/merge/update copies and deduplicate assets between different snapshots?
link