Y
Hacker News
new
|
ask
|
show
|
jobs
by
petee
121 days ago
Do crawlers follow/cache 301 permanent redirects? I wonder if you could point the firehouse back at facebook, but it would mean they wouldn't get your robots.txt anymore (though I'd just blackhole that whole subnet anyway)