Hacker News new | ask | show | jobs
by curio 5839 days ago
but their robots.txt says that you're welcome to scrape it:

  User-agent: *
  Disallow: /ac.php
  Disallow: /ae.php
  Disallow: /album.php
  Disallow: /ap.php
  Disallow: /feeds/
  Disallow: /o.php
  Disallow: /p.php
  Disallow: /photo_comments.php
  Disallow: /photo_search.php
  Disallow: /photos.php