Hacker News new | ask | show | jobs
by Whitestrake 3064 days ago
Most news sites, yes. They want Google's crawler to crawl their full text, and Google responds poorly to sites that treat their crawler differently than real search users, so if your Referer (sic) header is Google, news sites will often give you the whole article.