[c3o]

You are absolutely right: The mere fact that these publishers don't block Google using robots.txt and that they in fact spend a lot of effort optimizing their metadata so that link previews show up just the way they want them to proves that it's at least mutually beneficial, and not an abuse of their intellectual property.

[meh2frdf]

Because google owns the index, if they don’t allow google to index they get no traffic. Google is a monopoly, there is no choice here, no market driving competition.

[Zak]

The sources I could find with a quick Google (yes, possibly ironic) search suggest news sites get most of their traffic from direct visits, but Google contributes a substantial amount[0]. It seems to me that the news sites want Google to provide them traffic and simultaneously pay for the privilege of doing so.

The behavior of publishers in countries where they won this battle is telling: a when laws were passed in Belgium and Spain requiring aggregators to license even small excerpts, Google stopped, and the publishers didn't take very long to offer free licenses.

[0] Here's one slightly dated example: https://www.sistrix.com/blog/new-data-is-google-or-facebook-...

[meh2frdf]

They don’t want google or any other monopoly that is in competition with them to ‘give’ them traffic. They do want their services available and indexed on the internet. If the internet search market was evenly distributed among 5 search engines the I suspect this conversation wouldn’t exist.