Hacker News new | ask | show | jobs
by mikae1 1185 days ago
Good, but from previous experience they don't make these archives available to the general public.
1 comments

AFAIK this is mostly incorrect. First the crawler archive URLs to files in WARC format, which are uploaded to archive.org, which then imports the files into the Wayback Machine. For ArchiveBot jobs, you can find the WARC files for the completed jobs for any domain from the ArchiveBot completed jobs website. For other types of jobs, including custom ones like DPReview.com the process is similar but involves more machines due to their more distributed crawling process. It all ends up in the Internet Archive Wayback Machine though.

https://archive.fart.website/archivebot/viewer/ http://archiveteam.org/index.php?title=ArchiveBot

Woops, forgot the end bit. There is only one non-public archiving job that I know of, and that is for YouTube videos only.