Hacker News new | ask | show | jobs
by dymk 2637 days ago
Does the ArchiveTeam coordinate with services like Google+ for archival? Are the techniques used (such as spreading the load across 1000s of separate IPs) used for anything other than getting around Google's integrity services e.g. ratelimiting?

I'm really curious how such a large operation is legally pulled off, especially when services like Google+ intentionally try to make scraping difficult (and presumably for large offenders, will attempt a C&D).

2 comments

Sometimes the operators of a service will contact the ArchiveTeam and work with them to speed up the archival process. The file hosting service pomf.se did this when they were shutting down.

https://www.archiveteam.org/index.php?title=Pomf.se#Archivin...

They use the ArchiveTeam Warrrior VM, so they have access to many public IPs: https://www.archiveteam.org/index.php?title=ArchiveTeam_Warr...