Hacker News new | ask | show | jobs
by randomdata 4565 days ago
Depends on the intent. If it is user-initiated (like say a mobile formatted version of the site), it wouldn't have to be obey the robots.txt, since it is not a crawler, just another web browser.