Hacker News new | ask | show | jobs
by tardedmeme 32 days ago
It's not about robots.txt but yes, the owners of 538 can just send a cease and desist letter to get them all immediately removed. Many sites that don't want to preserve history have done this already.