Hacker News new | ask | show | jobs
by pflenker 35 days ago
A couple of years back I worked with a company which maintained specific data which was the main traffic driver on that page. Google approached them and wanted to pay for the rights to get the data and display it on top of the search results, a feature which was fairly new back then.

This was an interesting dilemma because it was very clear that the money was way less than the loss in ad revenue due to traffic drop, but it was also clear that if we wouldn’t take the deal, a more desperate competitor would, which would result in the same traffic loss but without the extra google money. So the company took the deal.

History repeats itself here, with the difference that instead of paying for the data, the ai crawlers simply take it for free.

10 comments

That's the problem with the current monopolistic system, the money won't go down the stream, it's like a dam. One big dam owned by a few people is worse than many small dams
I agree. For what it's worth, I think Google's business model is in trouble. They're putting in heroic effort to stay relevant during this shift to AI search, but how does their advertising model work when people expect summarised and accurate information instead of scrolling pages of ads?

The search space is actually not that unhealthy. There are competitors. They just lack the reach of Google. These competitor will become increasingly useful when our personal AIs plug into them to search for the right info.

That said, I think the old internet business model is dying. Why will people write articles and pay for hosting if they can't receive any advertising revenue? I think I'm okay with this. Before advertising decimated the free internet, people ran sites for fun. Maybe we return to that.

>> but how does their advertising model work when people expect summarised and accurate information instead of scrolling pages of ads?

1) because that fancy summarised results page will effectively be the ad. The results will push you towards google's partner sites or promote behavior benificial to them.

2) because partners may soon be able to pay to change the results of searches. Who won an election 20 years ago? That "fact" from the AI could depend on which answer has been sponsored by which political interest. What happened in china in 1989 ... how much money do you have?

3) the user becomes the product. No ads in search results, but our searches and online behavior tracked by them will be sold to whoever is most interested in what we do online.

>... I think Google's business model is in trouble.

>Google parent Alphabet profit jumps 81% in Big Tech earnings roundup

https://www.euronews.com/business/2026/04/30/google-parent-a...

Both could be true. I think Google is milking their main asset and tanking it, before the party ends.
This comment thread https://news.ycombinator.com/item?id=47954579 is relevant.

TL;DR: Google is heavily stuffing more and more ads into every monetizable SERP, which explains the ever-increasing revenue.

But my theory (expanded in more detail here https://news.ycombinator.com/item?id=47957708) is that its cash cow is heavily optimized towards its current anti-competitive, multi-sided chokehold of the online ad market -- look up the AdTech Antitrust findings around manipulating ad auctions, e.g. Project Bernanke -- which relies on ad volume.

With agents however, I suspect that volume will shrink drastically even if they stuff ads at each step of the conversation. Because it's only the final click that will matter, and those ads will be meaningless.

It's not clear that Google can charge enough for that final click to compensate for the loss of both, the value of all the ads that lead to it, as well as their ad auction manipulation premium.

Hanging my comment here, not blaming the parent poster, it's just relevant...

All the "AI search" I've tried is far less relevant than a proper search, using search engines without it seems much faster and more relevant.

Whenever I get AI search results, its summaries lack substance, are often just plain wrong, and so on. When I check, it's often usong the wrong aggregate data to return results.

Search engines these days, traditional ones, have been hobbled by targeting the lowest common denominator. They drop search terms, alias them (eg, Joe=Joseph,Josephene, Jodiene), and thus return endless bad results.

Using "web" search on Google, for example, putting all search terms in quotes, helps enormously. If searching for Joe Baker, you don't want aliased responses.

My point is, AI is yet another abstracted layer. It not only aliases things, it halucinates, and further, it doesn't even show the context it simply summarizes.

It's the next layer of crappiness.

As a side note, there os a product in my home country called the Swiffer. It is like a broom, but uses disposable sticky paper instead of broom bristles and a scoop.

Point is, it isn't very environmentally friendly. I weirdly see environmentalists using it, instead of just a good old broom.

And now I see environmentalists using AI for search. What? Why! The sheer additional power and cost, all the new datacenters, hardware, just for what??

Even of one believes it is mildly better (which it isn't, if you use traditional search properly), are you now replacing your broom with a swiffer?!

The amount of environmentalism that goes out the window, for convenience, astonishes me.

It is totally dependant on context. Ask about product purchases/prices and it will give you great results. Ask for basic data and it will quote wikipedia or government websites all day. Ask for a coding solution and it will spit out answers ripped from the corners of github. But i asked google about hydrogen-3 recently and got back a result saying that we were mining it on the moon. An entire space economy was invented because AI cannot tell scifi from reality.

The real test: dont test AI by asking it questions you already know. Ask it a proper question and then go out and find the answer yourself. That is where AI really falls down, the non-trivial stuff.

I’ve experienced the opposite. I get good summaries with links for original sources when something seeems off, which it almost never does. I also like that I can glance to see where the data came from.

Several months ago, I was getting hallucinations and silly answers, like Jaco Pastoreous being the bass player for Metallica.

but lately it’s been great. Especially for technical syntax stuff, like asking for syntax on a jq command, or sal syntax, or something trivial like movie casts members….

Edit: I also assume there’s now some pre-processing filter to retrieve answers from a cache or something, because I’m getting pretty long answers very quickly….

I've noticed something of a hybrid of what you posted and what one of your responders below posted.

For average results, that can have an acceptable answer that tends to the mean or societal mainstream views along some subject, the google answer works for me. For others, which are well unfortunately most of my queries these days, the google answer is brainwashing and propaganda. It feels little more than a continuation of the SARS-COV-2 era fact checkers which were actually failed attempts at propaganda machines for enforcing certain views as true while others were cited as "evidence does not suggest" (hand selecting the evidence of course means this is nothing more than confirmation bias).

I think the AI technology used in most LLM models is an acceleration of this- it will tout mainstream views or hand-picked views as truth while calling others false. And without an ability to search any more for opposing views, you get left with a really warped view of reality that might work for most. Though not for everyone.

>All the "AI search" I've tried is far less relevant than a proper search, using search engines without it seems much faster and more relevant.

I've had the opposite experience with Perplexity Pro. Yesterday it took just under nine minutes to deliver a detailed and accurate answer as to the nearest locations where I can buy Meta Ray-Ban Display glasses (at least 3 hours drive away FWIW) that day.

In the past several months I've spent countless hours using search engines with dismal results that weren't close to that.

Regarding a swiffer: it's a zero sum game. Where did the materials come from? The earth. Where did they return after use? The earth. The only thing changed was the energy used to create and transport them.

If conserving energy was important, none of these AI datacenters would be built, none of our items would be shipped across the planet from Asia. They would conserve energy and import raw materials only.

While I agree in principle, I think it was inevitable. People been gaming SEO for so long that even with judicious use of search operators, it was getting harder to find the things I wanted (probably drowned in a sea of spam in such a way as to fall outside of the optimized search index). It _looks like_ the AI overview does not have this problem (yet...).
Anecdotally, I find Google's AI Search results to be genuinely helpful much of the time, and the box isn't so intrusive that it prevents me from digging into literal search results if I am not satisfied with the AI summary.
The new business model is misinformation sold by the highest bidder. Ask it what's the most reliable kind of computer, it answers with whoever paid the most this week.
That's a pretty sinister system when the dam builders are suing the work of those downstream to build their damn. What happens when everyone downstream has been starved to death?
The very people calling the shots have so far been the most removed from the consequences of their actions. They have no incentive to be responsible or considerate of others.
Obviously the pitchforks come out before then.

But! Does Musk have his robot army or not at this time? That will determine how the pitchfork mob turns out.

That's why if trickle-down economics were real, its proponents would also support antitrust enforcement
This reminds me of Walmarts squeezing strategy with all the manufacturers. Business with us at the price we say or out of business.
Yep, this is exactly why some companies simply don't work with Walmart.
But ultimately that strategy is good for the consumer right?

In this context, if Google is going to give me the recipe without having to scroll through the story, that seems like a win to me.

The ad-revenue driven Internet of web 2.0 is finally dead and I'm not sure I'm all that sad.

But google won’t give you the recipe. It’ll give you a pretty piece of text that resembles a recipe. You’ll only find out it’s not a recipe when it fails to produce a cake.

But then, the sites it's training on are starting to do the same thing, so maybe it won’t matter. Just last night, I pulled up four sites with “gluten free almond cake” recipes. One specified less than 1/4 the flour it would have needed, and another didn’t have any butter in the ingredients list. I had to eyeball the median and tweak from experience to actually get a bakable cake out.

You can buy recipe books at brick and mortar stores, and they're mostly not AI slop yet.
I'm not going to drive to the bookstore only to find out the recipe I need twice a year is not there because the local cousine doesn't even know the ingredients.

Oh, maybe I should drive to the major city and check several bigger bookstores?

Or order it online? But that would be from the third country because this one is weirdly blind to variety. That will be €20 for a book and some €10 postage.

You're sure I can just get my recipe from the store?

My partner recently bought a receipe book that was probably 100% slop, from the recipes themselvea to the images for each dish.
Completely unrelated to AI, I buy and always bought recipe books in second hand bookstores. It has this vintage vibe, and not everything has to be kale. Downside is that some ingredients aren't easily reachable, but it doesn't happen that often.
Online grocery shopping service I use has added recipes to their website. Not obvious slop at first reading but then you see stuff like add 600g of carrots and 100ml of water to make a quite watery soup according to the picture.
We're already post dead info.

The only solution is to find recipe books that were printed in previous decades.

Which is ironic, given that Google's entire value proposition (to users) was extracting signal from noise...

... and now it's come full A/B-advertising-optimization to being useless at that, when the need is greater than ever.

Imho, Google's greatest failing was missing how its own incentives warped creation of new web content, and failing to account for that strategically -- it turned the web into something it can't itself usefully parse.

You don't need butter to make cake, it ends up lighter and softer without it
You need to compensate for not adding it though. The recipe without was literally the same ingredient list as the site it copied, just missing a line.

And almond flour does its thing by carmelizing in combination with butter and sugar, turning your whole cake into a sort of giant macaron. You can’t pull it off without any one of those things.

LOL, that's some confident pan-cake misinformation.
> But ultimately that strategy is good for the consumer right?

No. It may be good for the consumer right now, but not ultimately (and on top of that, I would argue that reducing everyone to a consumer is already the wrong framing -- you need to ask what's good for the citizens). Having ten competing supermarkets with various interweaving supply lines is ultimately much better than having one giant supermarket, because that one monopolist is able to squeeze both consumers and suppliers to the detriment of both.

Do you think Walmart’s prices are higher than you’d pay if there were just ten competing stores with the same product mix, each of which did not have the leverage to squeeze suppliers?
It doesn't matter what Walmart's prices would be -- because there'd be nine alternatives offering their products at different price points. Also, you seem to be implying that suppliers are inhuman and deserve to be squeezed (but Walmart somehow deserves all the wealth it extracts as a middle-man, go figure), which is exactly why I said that focusing on consumers only is a myopic view of the world.
Please please please learn the basics of monopolies.
While I appreciate the condescension, tell me how Walmart is a monopoly?
Without some way to generate revenue, people aren't going to publish recipes (for Google to scrape into their AI.) Maybe we could live without more recipes being fed into the machine, but there are many other types of content that will suffer the same fate.

It would be nice to find something better than an ad-revenue driven web, but I'm not sure this is it. We'll find out I guess...

> people aren't going to publish recipes

Sure they are. I can attest that musicians will gladly publish their music even when no recompense is offered. Surely culinary artists are the same.

Some people will. The vast majority won’t. You are not playing for an audience, only for crawlers, so what is the point?
Those who won’t were doing it for the money. Those who continue are those who do it for passion, or those whose recipe is just a way to attract people to their business (e.g. kitchenware company). I don’t think it is necessarily bad, the quantity will decrease but the quality may even improve
We just won't get countless recipe websites where you have to scroll, scroll, scroll through slop about someone's day to read a scraped recipe that every other website has.

This is just disruption.

Does the current Google search results indicate that they will be any different?

It’s also not disruption if your product relies on the output of the industry it kills. What will AI train on when it destroys the economics of sharing information with others?

No, because now Google controls entirely what you see. They could decide to show you the recipe after all.

Also, at some point even the ad-laden websites will die, and then Googles sources will be extinguished.

> But ultimately that strategy is good for the consumer right?

No because it's killing competition and becoming an even more obvious monopoly. Then at any occasion they have to choose between consumers and profits, they'll do what shareholders want and increase profits.

> But ultimately that strategy is good for the consumer right?

No. Temporarily it’s good for the consumer. Ultimately it is bad for the consumer, because as prices drop, so to does quality.

It's not uncommon for free things to be higher quality than cheap things, especially when we're not talking about physical goods. Think hobbyist vs hack. Selective pro bono vs quantity over quality. The former describes old internet while the latter describes much ad-supported internet. I'm not saying cheap is better than expensive, and I'm not saying everything works this way, but I do think many things do, especially for pure information that doesn't have a major capital cost associated.
You can find many ad free recipes in the cookbooks at your local library. They're likely of higher quality as well.
Not really good for consumers in the long term. Creates a monopsony (monopoly on the buyer side instead of seller side) in the supply chain, with all the same negative externalities to competition.

Same reason its not good to have a "company town" where 1 company is the major employer to 50% of the workers.

Not even the summary of the recipe, I think a lot of the "value" AI provides is decluttering the interface. All these perverse incentives conspire to make it undesirable for web publishers to simply show you a cake recipe. They need to meet SEO metrics for being higher on googles index, they need to monetize with ad and chum boxes. They need to distract and dazzle your senses. In the end, AI strips all these anti patterns out and shows you the meat. Or the "summary" of what should have been there all along. The "AI" isn't learning about what the words mean, it's stripping out cruft and filler that was deliberately added!

I'm not saying LLMs are worthless, but I'm saying if you had a magic browser add-on that simply stripped the BS out and showed you the relevant content, it would meet the use cases of the majority of people using AI (regarding search).

Said differently, Google are bringing you a solution to a problem they (largely) created.

except this isn't a solution. taking away sources just creates another problem down the road. there's no information to crawl from if no one is building websites to crawl from.
it's because both google and walmart have too much market power
This is tough for the manufacturer, but great for the consumer.

I think it's a good tradeoff.

Real-world Prisoner's Dilemma.
It always comes back to game theory haha
>History repeats itself here, with the difference that instead of paying for the data, the ai crawlers simply take it for free.

This discussion was broached originally when discussing whether or not search engines and aggregators had any compensation obligation in respect of news articles. This was a hot topic in the IP and policy circles for a few years.

When the Canadian government attempted to create a mechanism to compensate content creators for the scraped content, there was widespread outrage from tech circles, despite the same community agreeing, across extensive policy discussions, that action had to be taken to prevent this universal man-in-the-middle value capture by search engines.

I've had fairly extensive discussion with the individuals involved in the academic, policy and internal industry analysis of the issue. Watching industry agree to address the issue, then aggressively spend to shape public narratives in public was eye opening.

The recent shift into "AI is obviously going to hoover up all your data and there's nothing you can do now that the theft is laundered through an LLM" is just the latest example of the same trend of short sighted capital-over-everything decision making we've become used to in jurisdictions that have dysfunctional legislatures.

A similar dilemma presents itself when blocking AI spiders.

You’re free to block them, but the websites cloning your content won’t. So either way they’ll get the content they’re after.

Worse, when/if the time comes that LLMs source their claims they’ll refer you to the websites that cloned your content.

That doesn't feel like a repetition at all? You said that the first time there was "traffic loss but without the extra google money", but that this time there's no extra google money either way.
The part where data providers lose traffic because their own data is displayed directly on the google premises is what repeats.
Then we go and pay to Cloudeflare or other cloud providers you host sites with to prevent Google from scraping.

Let the big guys fight it out.

There is money in protecting the websites. If you host with OVH they are interested in you making money so you can pay them.

This won't work because Google just pays Cloudflare to enable scraping.
Any evidence for this claim?
I don’t see where they charge for it.
Ok but how would people find your website then
How are people finding your website now?

You are on top of Google search results or something? I think I am on some last page that no one will click like 2nd page of results.

You put it in your hn bio /s
Oh the joys of the free market. At least Google didn't have to suffer any red tape or taxes, and we all should deeply care about Google's suffering.
But like why would anyone make original content or websites anymore if google actively works to ensure you make no money
Why? To sell you something. If the traditional web incentives were to disappear, the internet will become just ads, selling things & work.

The web won't die. It will just become even more commercialized than it already is. You might recognize most of the things listed.

Selling things: Restaurants, hotels, local businesses; Ads: more of the same

Work: a) Projects people collaborate on and need to share information b) Writing articles to boost a resume

And maybe sharing what people do, they do it for free, see social media.

"Nice data you got there, it'd be a shame if something were to happen to it"
The fact is that internet is already "tech giants own realm": the power is way beyond public imagination and affects all of us in real life on daily bases, but there are still people thinking they are not the "evil one" here.