[whywhywhydude]

Going a step further, I think google should stop crawling websites that are paywalled. When I search for something, I want to see results that I can click on. Not some snippets from NYT, WSJ, Bloomberg and others which are heavily paywalled.

[renewiltord]

You can easily write this as a Chrome extension if you so desire.

[matbilodeau]

At least the "archive" workaround still works

[pessimizer]

It's piracy, though. I'm sure plenty of paywalled outlets would be happy to see it gone.

[jupp0r]

It's not piracy, they explicitly allow this.

[kccqzy]

The open web has deteriorated to such an extent that either information is paywalled or free but has commercial motivations behind it (affiliate links, sponsorships). It turns out there's little free, non-commercial, high quality content on the web.

[chongli]

It turns out there's little free, non-commercial, high quality content on the web.

I disagree. I think there's a lot of it out there, in the form of blogs and small forums. It's just really hard to find, like mining for gold in the Super Pit [1]. You need to sift through mountains of rubble to find tiny amounts of gold.

[1] https://en.wikipedia.org/wiki/Super_Pit_gold_mine

[shadowgovt]

> turns out there's little free, non-commercial, high quality content on the web.

Well, sure. Who pays for it?

[kccqzy]

No one. That's why I commented to make the point that Google should not refuse to crawl commercial content.

[pipo234]

True. There is some great content (blogs, OS software, books, p2p networks, public libraries, academia, government), but by and large that wasn't created with monetary incentives.

Could there be a way to sustainably make enough money from visitors without making it all suck?

[kccqzy]

I think there really needs to be a micro-transaction mechanism on the web, but unfortunately it was needed ten years ago and there still isn't one.

[golem14]

Yes but the crypto folks didn't do that, they were busy chasing nfts.

[chroma]

There have been a lot of ridiculous things in cryptocurrency land, but what's wrong with Basic Attention Token? It has been around for six years and seems to solve the problem of paying content creators via microtransactions. The only wrinkle is that finance laws force everyone to verify their identity with a government-issued ID before transacting BAT.[1] This is a problem for all microtransactions, not just cryptocurrency.

1. https://support.brave.com/hc/en-us/articles/360032158891-Wha...

[pipo234]

...And then we're back with the underlying conflicting interests surrounding governments guarding their citizen's privacy. Yes absolutely we want all our citizens to always use unbreakable and untraceble communication channels. Except when law enforcement needs to surveil and pry on the bad guys.

And where it's impossible to tell difference from a (pseudo) random stream of numbers.

[whywhywhydude]

It’s certainly possible. Wikipedia is free. It won’t be too hard to replicate wikipedia’s model to create a Reddit replica.

[chroma]

Google isn't circumventing paywalls. Most paywalled sites whitelist search engines so that their content gets crawled and more people visit them.

[carlosjobim]

When I search for something I want the best results for the query, and sometimes those are behind a paywall.

[nonameiguess]

At a certain point, if you want a subscription service, why wouldn't you just do something like suscribe to Bloomberg News, then get all your news by going directly to their site rather than going through a search engine or aggregator. If you're looking at an aggregator, inherently you want to see many possible sources, including ones you may only read once a year. Nobody is going to subscribe to hundreds of separate sources individually just to read them once in a blue moon.

Ironically, the predatory and terrible academic journal industry is probably the only thing out there right now that comes close to getting this right. Rather than expecting anyone to subscribe to each journal individually, they give a bulk subscription to an entire publishing service that then grants access to many journals.

If someone out there offered a $20 a month service that granted access to Bloomberg, NY Times, Wall Street Journal, Economist, WaPost, Financial Times, all in one, I'd gladly buy that. But there is no way in hell I'm subscribing to all of those separately. Even if the aggregate price was cheaper, I wouldn't want to do that.

[carlosjobim]

1. There is much paywalled information online that is not in the form of a subscription, but instead one-time fees.

2. Just because you paid at the paywall, doesn't mean you have to subscribe for life. You can pay to get the information you need and then instantly cancel any subscription.

> If someone out there offered a $20 a month service that granted access to Bloomberg, NY Times, Wall Street Journal, Economist, WaPost, Financial Times, all in one, I'd gladly buy that.

PressReader is pretty much this, although the price is $30 and not $20.

[JoshTriplett]

Checkbox, unchecked by default:

[ ] I want to see results from sites I'd have to pay to access

[kccqzy]

The obvious problem to this armchair expert "solution" is that Google doesn't know what I am already paying. I pay NYT for a subscription, but Google doesn't know that. For obvious privacy reasons users don't want to tell Google what sites they already have subscriptions with. And I don't even log in to Google to do a search so there's no place to store that information even if I actively wanted to provide that to Google.

[JoshTriplett]

I wasn't presenting it as a perfect solution. I was observing that many people don't want to see paywalled results, some may want to see all of them because they may choose to pay, some may want to see them because they plan to use a paywall bypass, and some as you pointed out may want to see the subset they already pay for but not others. As a first pass, a binary approach seems better than nothing, and is simpler to provide than a more complex user-subscription-specific solution.

[pipo234]

I'd pay a little if we could get rid of the pop ups and cookie banners, advertisements and click bait content. But after 30 years we're still missing the infrastructure for micro payments.

For news sites and netflix we now have subscriptions shielded by paywals, which really is incompatible with hyperlinked sites or search engines. Even if you subscribed to 1000 services, the experience would probably be horrible. The internet was designed to be free, but evidently that's not a good business model if you want to make a living.

[carlosjobim]

> But after 30 years we're still missing the infrastructure for micro payments.

Sad but true. The closest we have to a solution right now is PressReader, which is just too expensive in my opinion at $30 per month.

[pipo234]

Micro payments only make sense if the intrinsic transaction costs are (near) zero. Like reaching for your wallet to pick a coin to give to a homeless person.

With trusted third parties or block chains, the transaction costs are unfortunately much, much higher, especially initially without proper scales economics.

[carlosjobim]

It could be as simple for the user as a button in your web browser to donate or pay for the site currently open in your tab. And a third party intermediary that once a month collects payment/donations and distributes. The problem with Visa/MC is the minimum transaction fee of 10-25c

But a lot of work and planning would be needed to get anywhere with such an idea.

[admax88qqq]

If I'm not willing to pay for a paywall then the "best" content is not behind a paywall, because I won't read it.

My definition of "best" includes my ability to actually read the content under my terms.

[carlosjobim]

If I ask you what is the best restaurant in town, will you answer that it's your mommas house because there you always eat for free?

Cost has nothing to do with determining the quality of a search result, and search engines shouldn't discriminate against paywalled content. But I think it's a good idea to let users like you check a box to hide paywalled results.

[chongli]

There are millions of websites out there that are free for anyone to read, including this one! Restaurants that serve free meals are not the norm, so this analogy doesn't make a lot of sense. If every website was paywalled and required a subscription, like cable TV channels, then you might have a point.

[carlosjobim]

People use search engines professionally and not only for entertainment. There is an icebergs worth of important and valuable information online behind paywalls, not only articles or news. Information workers use a search engine to find the information they need, pay the cost if it's paywalled, and then cancel any subscription after getting what they needed.

Long gone are the days of "surfing the web", when most of us spent our time online just randomly browsing around.

[chongli]

I don't see why there couldn't also be a professional search engine. Academics have Google Scholar which is an amazing resource for them. A search engine that brought up high quality resources for professionals would seem to be pretty useful. It could potentially even have a single subscription to unlock all of the sites in a network, rather than individual paywalls at every site.

[admax88qqq]

That's an insane take on what I said.

No, I won't answer my mommas house because you can't just show up and eat there as a restauraunt. It doesn't fit the search criteria you asked for.

If you ask what the best restauraunt in town is, I'll give you a restauraunt.

If you ask what the best restauruant in town is, I won't send you to a place where you can buy a "best restauraunts in town guide" for 5 bucks, because that's not what you're asking for.