Hacker News new | ask | show | jobs
by WorldMaker 3172 days ago
You don't necessarily have to redirect, but you should at least include `<link rel="canonical" href="..." />` (as given example StackOverflow does) so that search robots and other website (scrape and/or API) clients know which one is the canonical path, to avoid duplicate efforts.
1 comments

That only works for some crawlers. Certainly not for users. Meanwhile, everything obeys redirects.

Since you bring up Stack Overflow, notice that they do the canonical redirect. Change the title in the URL and you'll get redirected.

Yes, the best approach is probably both, but it is crawlers that it matters more that they know the canonical paths more than users, and a crawler ignoring rel="canonical" is likely not much better than/as buggy as a crawler ignoring robots.txt; it's a specification they can ignore at their own peril.