[matheusmoreira]

>Many paywalled sites will let googlebot index their content but block other crawlers.

Doesn't that infringe upon Google's own rules? I always thought Google didn't like it when sites served its crawler content that's different from what users get when they follow Google's link.

[fauigerzigerk]

That's why many paywalled sites give you a few free articles per month if you're coming from a Google search results page.

But it no longer works at all sites. Maybe the rule has been dropped now that paywalls are becoming more popular (with publishers that is)