Hacker News new | ask | show | jobs
by mzmcbride 2533 days ago
This URL was returning "Not Found" for me as well, but with an HTTP 501 status instead of the traditional HTTP 404. After investigating, the issue seems to be cookie-related:

  $ curl -sIXGET 'https://www.reddit.com/robots.txt' -H 'cookie: rseor3=true;' | head -1
  HTTP/2 501
Compare with:

  $ curl -sIXGET 'https://www.reddit.com/robots.txt' | head -1
  HTTP/2 200
1 comments

Thanks, works from incognito mode in browsers also then :)