[burnte]

> What scares me about this new AI mode thingy

What scares me is the rampant inaccuracy. In my experience, the AI responses are wrong about 65% of the time. I just did a search today about an error talking about a disconnected link between apps, and Google AI result summary told me that the error was related to my pulling a USB drive too quickly in windows. The ONLY word similar to my query and that AI response was the word "disconnect". Everything else was clearly about the SaaS apps.

I have people coming to me, asking me questions, then telling my Google told them something else, so now I have to waste time convincing them that it's wrong. Over the past 2 years AI has done nothing for me but complicate my work life.

And of course, this could be because the model is crap, but it could be because they want me to keep refining my query over and over for more ad views. Either way, it's a terrible experience.

[sanitycheck]

Yep. For years we've been telling people to 'just fucking google it', and now when they do they're getting bullshit AI answers.

Worst thing is, some of these bullshit answers will be medical, some of them financial, it seems pretty certain people are being harmed.

[pants2]

Yeah the Google AI results are more dangerous than ChatGPT, not only because it uses a smaller model but because Google's knowledge graph used to deliver very accurate and authoritative information but now that's been replaced by a stochastic system in the same place, so people are used to trusting it.

[Robotbeat]

I think we’re getting what we deserve by snarkily telling people to Google stuff instead of answering accurately. Google results have never ever been pure accuracy

[tveita]

The point of LMGTFY is to land people on either the official documentation or a curated site like Stack Overflow. Google used to be able to do that reliably.

With the power of LLMs you can Google a standard library function and get an inaccurate summarisation of a Reddit discussion where neither side knows what they're talking about

[Robotbeat]

Stack Overflow and Reddit for years have told people to just Google it. And then the Google result is people saying to just Google it, instead of actually being helpful.

[awesome_dude]

To be fair - for all of those years Google has been serving up some atrocious results - remember when googling health symptoms got you rabies or pregnancy.

There's even the meme where people ask if the code was the result of a stack overflow question, or answer

[RyanOD]

It seems to me one needs to consider the complexity of the question they are asking before searching it.

To stick with your post, consider people asking medical or financial questions. For a wide variety of reasons, many of such questions don't have an answer. In such cases, AI is still going to take a crack at it. AI shouldn't be blamed for "bullshit answers" to such questions.

Before using AI, I think people should stop and ask themselves, "Is there really a single answer to this question? Is AI the right choice?"

[gsk22]

The problem is Google's AI results get even simple factual questions wrong all the time.

Earlier today, I searched "pixel 10 wifi 7" because I was confused that GSMArena showed my Pixel 8 supports Wifi 7, but the Pixel 10 only Wifi 6. Gemini confidently claimed that the Pixel 10 does support Wifi 7 -- but that's not true at all. Only the Pixel 10 _Pro_ supports it, as I discovered when actually reading the non-AI search results.

And this is a question about a Google product!

[stephen_g]

I had a similar thing when I was gooling a few days ago, I can't remember exactly but it was like "why does [product] not support [feature]" and the AI summary was confidently wrong, saying "The product does support [feature]", which knew was completely incorrect, and I did find a Reddit discussion or something in the actual results with discussions that were actually about what I was looking for!

It's really depressing how bad things are getting...

[kristjansson]

It’s hilariously persistent in this, esp. for anything even slightly divergent from the beaten path. Discount everything the AI box says about emacs to zero.

[thedougd]

Admittedly I’m unsure if it was Google or DuckDuckGo. I switch between both. I quickly asked the in search AI for a UTC time conversion like a lazy fool and it got it off by almost a day wrong.

[ultrarunner]

I avoid any asking any agent a fact-based (especially math) request. It's a great compression algorithm and a great language generator, and I guess the intersection of those two things is "an answer". Calculation doesn't intersect.

[varenc]

My google search for 'pixel 10 wifi 7' immediately shows the right answer. (10 Pro and 10 Pro XL support it but, but base Pixel 10 only supports Wifi 6E).

Though the inconsistency of results between users is definitely another frustrating thing.

[RyanOD]

Ok, fair. Hard to understand why it would get that wrong.

[codebje]

Because LLMs aren't sentient, they don't draw on facts, and they don't have nuance. The answer given is similar to answers you might expect to see for similar questions.

It's really amazing we can make machines do that, and it's really depressing that we think a stochastic bullshit machine is going to give us something we can rely on.

[Robotbeat]

Or… the default LLM Google uses for search has been quantized to s**. Ask a proper Thinking model, with browsing enabled, and odds of a correct answer are much higher. There’s been substantial improvement in AI in even the last year.

Ask a human a question like this, and they also have a chance of getting it wrong, even when confident.

[andrekandre]

its bad in dev as well... i've seen llm code review bots tell me things that are flat-out not true; this like "this wont compile because windows 11 doesn't exist" like wtf am i paying for this again?

[Groxx]

They are this wrong about everything, but you don't usually notice it when using it to look for things you aren't an expert in. The default stance really does need to be "do not trust, verify" at all times.

They can still be useful, e.g. they're significantly better at finding "I want a thing that does x but not y and it must be blue, or maybe two things that can be glued together to do that" than classic search. But they'll routinely miss extremely obvious answers because the related search it ran didn't find it, or completely screw up what something can actually do. Checking more pages of results by hand or asking humans who know even a little about those fields is still wildly more useful... but they're absolutely slaughtering the sites where people do that, by stealing all the real traffic and sending DDoS-level automated requests.

[RyanOD]

How can you say they are wrong about "everything"?

I built a retro game clone once and I used that project as a way to try out AI. While it wasn't perfect, it definitely wasn't wrong about everything. I'd go so far as to say it was probably correct (or damn close) 75% of the time.

I see people on HN all the time saying AI is terrible, but that just isn't the experience I'm having. I'm willing to admit it may have something to do with me not being able to recognize I'm being fed bullshit. Or, I may be asking really simple questions. Who knows? But AI seems like a pretty useful tool for average people.

[Barbing]

I’d make assumptions about how the cheapest and fastest possible flash model optimized for being extra cheap and extra fast would get something wrong based on its limited context (which can be very incomplete summaries of search results)

[bitmasher9]

I often have the expensive models give relatively simple inaccurate answers, even when they cite sources that directly contradict them. The error rate is lower, but you can’t have confidence with llm answers.

[pesus]

It somehow seems to interpret whatever sources it's grepping as the exact opposite of what those sources say fairly often. I've lost track of how many times I've clicked on the sources it cites, and every single one is in agreement, but the AI claims the opposite.

[facemelt2]

Did you just agree to a stranger's counterpoint on the internet? This post should be in a museum somewhere

[SequoiaHope]

The simple answer is that these systems are very bad at telling the truth reliably.

[thwarted]

When the default "search" results are AI, it's difficult, if not impossible, to "choose", since Google is pushing the AI so hard.

[RyanOD]

In watching the demo, I didn't come away with the impression that they were removing search results. Yes, they are pushing AI hard, but users can still opt to use Google in the more traditional way. Unless I misunderstood the demo, it's definitely possible to choose.

[makeitdouble]

"possible to choose" doesn't get us much.

An interesting aspect of this is the decrease in quality feedback on th organic links. If most people never get down to the actual links there is very little to tell which ones were good or if they had any relevance.

There is also that less incentive to properly maintain the search algorithms to fight SEO and spam.

For all intents and purpose, organic search results have been given a death sentence and are just waiting for the last moment.

[RyanOD]

Organic search dying was my first reaction too. But, who knows...this wouldn't be the first time I've heard that.

[Barbing]

They are showing billions of people a big bold answer at the top of all their pages.

What a wildly irresponsible company

[rbits]

Go to Google right now and search anything. What is the very first thing you see?

[autoexec]

> one needs to consider the complexity of the question they are asking before searching...consider people asking medical or financial questions...many of such questions don't have an answer. In such cases, AI is still going to take a crack at it. AI shouldn't be blamed for "bullshit answers"...people should stop and ask themselves, "Is there really a single answer to this question?

It's a bold position to say that it's the users fault for being lied to by Google. There isn't a "single answer" to most questions. It's still Google's job to provide answers that are accurate and reflect the best information available on complicated topics. That's what they're trying to sell us anyway. When google's AI can't live up to the hype "You shouldn't be asking AI such difficult questions" is not a great response, especially when people are just trying to get web search results and AI is suddenly interrupting with an opinion nobody asked for.

[melagonster]

In past, people can trust Google. Now we should teach children don't trust "search result" from Google.

[freeone3000]

I asked it “how can I tell if a spray paint can is empty?” And it told me that the paint can would no longer rattle.

[xorcist]

It's nice that Google's AI summary always lists its sources. It's less nice that those sources more often than not do not corroborate the summary. It often seems to be a few random links thrown in there for good measure.

I have no idea why this is, but it is impossible that these links are primary sources of the data, if such things even exists at all. In which case, why list them?

It is certainly seems possible that the actual sources of the data is the output of some other LLM.

[consp]

> It is certainly seems possible that the actual sources of the data is the output of some other LLM.

My guess is you can see this happening with the bots on Reddit where they are refining the answers to one certain thing, often getting two or three the same responses in a row from different users because they have been enforcing themselves by digesting the output of other bots. Waiting to see when they cut down the sentences and start talking garbage.

[kyleee]

I’ll bet they intentionally obfuscate so people can’t find the actual sources of info used for the answers

[arcanemachiner]

Reminds me of this gem:

https://www.reddit.com/media?url=https%3A%2F%2Fi.redd.it%2Fn...

[thincopperfoil]

Straight of of x-files s02e03

[godelski]

  > What scares me is the rampant inaccuracy

What scares me is the massive incentivization to manipulate the results.

With AI ads you get all the power from big data aggregation, the trust/framing of an authoritative voice, and cheap personalization that specifically optimizes for what convinces you. It's too powerful. Even if it only works a small percentage of the time we're interacting with these things so frequently that a small percent is a large number. They're already feeding user profiles into these machines and there's explicit talk about having the LLMs optimize ad campaigns. It's already dystopian if it's ads to get you to spend your money, but people seem to dismiss that. Do we not care that this is also being used in the same way to convince you to believe certain things? To join certain political organizations?

Yeah, these things help me write more lines of code faster (if we include all the lines from our design docs) but I don't like the idea of pointing a supercomputer at my brain and someone else using it to try to manipulate me. That's not a game I'll win. It's not a game you'll win either.

[HDBaseT]

The built-in Search AI is fucking braindead and people constantly come up to me "Google said xyz" and I just have to turn around and say "I do not care what the Google Search AI said".

Whatever it says is a waste of time 99% of the time. Although people believe it, or consider it worthwhile majority of the time because its so simple to use. It's always there, always instant and appears at the very top.

I would much rather people shove a question into a locally running Qwen model and tell me what it said rather than use the nonsense search model. I hate it.

/rant over.

[redml]

accuracy hasn't been their priority for a while now - they just want people to click on ads

[dzhiurgis]

Google has been around for a quarter of a century. People are still incredibly dumb and will believe whatever they like.

[wvenable]

Free AI's are dumb. Extremely dumb. The Google AI result is dumb on purpose -- being smart requires more compute.

[WarmWash]

Can you share the query?