[danso]

Why would it be penalized by Google? One of the factors in the SERP algorithm is how accessible a page is when someone clicks-through from Google, including load times, page-covering ads, and so forth.

Google also punishes pages that show one thing to the crawler and another thing to human visitors who click through on the search results, i.e. "cloaking" [0]. That is not the case here.

edit: Maybe your question is actually: why does HN -- and all other non-Google sites who link to WSJ articles -- continue to link to WSJ articles when it provides a bad experience for the user? HN admins have made their position clear that the content of WSJ articles are worth going through the workaround.

[0] https://www.searchenginejournal.com/17-ways-to-get-de-indexe...

[tarunm]

Well I did some clicking around and came to know that this falls under their First Click Free [0] option. The details sounds more unfair to me. Example -

'A user coming from a host matching [www.google.] or [news.google.] must be able to see a minimum of 3 articles per day.'

Isn't this a net neutrality violation? Because of their monopoly, they are able to force a work around to get access to content and a better UX for their visitors at the cost of other referrers. A more fair/neutral program could be a 'limited subscription' type access where the content site allows a limited number of articles to users (from any referrer), Google gets access to content for indexing and tags the content appropriately as 'limited access'.

[0] https://support.google.com/news/publisher/answer/40543

[danso]

Net neutrality applies to services like broadband, which are argued to be common carriers. If Comcast is the only Internet provider in my area, I have no easy alternatives if Comcast decides to block all Time Warner affiliated content. Just like it's not trivial to deal with PG&E arbitrarily shutting off my gas and electric.

This isn't the case with search engines. If Google were to blacklist wsj.com, I have many easy options to get to wsj.com. Which is why Google and any search engine has wide latitude to heavily discriminate against content. That's literally the point of PageRank, or any ranking algorithm. Where they have gotten in trouble is when they're accused of promoting Google-made content above other content (especially when the Google content consist of scraped info from other sites).