[tyingq]

They should, but many good sites just don't, for whatever reason. Even ones you know are run by smart people:

  $ curl -I -X HEAD https://redis.io/
  HTTP/1.1 404 Not Found

For this app, being more forgiving is a better idea.

[jaredsohn]

https://www.stumbleupon.com had a similar issue.