Hacker News new | ask | show | jobs
by brokensegue 3007 days ago
wayback machine's archive of reddit isn't perfect but it works. just give the IA more money
1 comments

Yes, but much of the Wayback Machine’s reddit content was specifically targeted and scraped by ArchiveTeam, who are volunteers that seek out at-risk content from the web and make sure that it gets into the Wayback. In the past few years we’ve specifically tried to go after sub-reddits that we thought were newsworthy and/or at high risk for deletion. But there’s no way we can get all of it.

But you can help! If you have extra server space/bandwidth or you can spare $40/month, we can add more pipelines: https://www.archiveteam.org/index.php/ArchiveBot

Source: am ArchiveTeam member, run various pipelines, have scraped sub-reddits ranging from The_Donald to the cryptocurrency worlds to darknet markets.