|
|
|
|
|
by mzmcbride
2533 days ago
|
|
This URL was returning "Not Found" for me as well, but with an HTTP 501 status instead of the traditional HTTP 404. After investigating, the issue seems to be cookie-related: $ curl -sIXGET 'https://www.reddit.com/robots.txt' -H 'cookie: rseor3=true;' | head -1
HTTP/2 501
Compare with: $ curl -sIXGET 'https://www.reddit.com/robots.txt' | head -1
HTTP/2 200
|
|