[Thanemate]

My non-tech savvy mother started reading the stuff the Google Search AI answers to for some searches, and she's already fed up with it saying whatever. To her it doesn't matter that the "AI can make mistakes" because (in her own wording) "if it's faulty, don't answer".

There's a difference between "linking to a source that may be incorrect" and "you providing the text that's blatantly wrong", and Google seems too big to care about it.

[GolfPopper]

Google only cares when something gains traction (and thus potentially hurts its bottom line". For example, it was answering "How many p's in Google" with "There are two p's in Google"[1] for long enough for me to get non-technical family members making a joke out of it. Google fixed it, only for it then to briefly tell you there were no p's in 'Alphabet' either.

Those particular bits on nonsense seem to have been stopped for now, but let's not fall prey to Gell-Mann amnesia. The only problem Google has fixed is "our LLM was hurting our reputation in this specific case". They have not, and likely cannot, fix the underlying problem.

1. https://techcrunch.com/2026/05/27/why-googles-ai-cant-spell-...

[ragequittah]

I fail to understand why people think LLMs working in tokens (and thus having a hard time counting letters) is such a huge gotcha. Ask it to write you a bash script that counts the amount of p's in a word you input it'll do it fine. Then ask it to write that script in C, JavaScript, Lua, and a few other languages. Then ask your family members to do the same. The gotcha can go both ways.

[shakna]

LLMs are directly marketed as not requiring understanding of how it works.

The PR is "Its magic!"

So people try to treat it like magic, and don't like when it detonates a fireball in a room that is too small.

Not requiring technical knowledge is the selling point. That it doesn't need you to think.

[ragequittah]

Questions like this are irrational though. Nobody wants to know how many of a type of letter are in a word. Ask it a real question from your real life that needs a real answer it's fairly magic at getting the correct answer. And if you don't think so at this point my guess is you simply haven't used it enough.

[shakna]

Counting letters in words is absolutely a real question. Its trivia!

Half our TV is made up of trivia, and being able to generate trivia answer/question pairs is something both TV, and pubs, will try to do.

[ragequittah]

Imagine seeing this as a jeapordy question. What is 2 Rs.

LLMs have some limitations same as humans. The limitations are very different between the two which is why LLMs are very useful to us. I personally can't write a perfectly functional fairly complex webpage in less than 30 seconds, but I can count letters. I'll let the LLM make the webpage and keep count of the letters for a trivia contest if needed.

Also get the latest version of chatgpt (extended thinking) to make you a 'tiptop 100% verified Trivia stack for (x TV show)' and you'll get better than what most pubs will come up with. I did this for the simpsons when someone said it couldn't be done because that sounded like something an LLM would be excellent at. Turns out I was right.

[bigstrat2003]

Because it proves they are not, in fact, intelligent and do not, in fact, have any understanding of what they are doing. It's really obvious what people think it's a huge gotcha: because it is.

[shevy-java]

Yeah. AI slop is lying to people. When I realised that I disabled it. Thankfully there are browser extensions that do that easily.

People call it hallucinating. I think it is lying. Google etc.. became a huge liar. All those AI slop companies are lying to the people now.

[somat]

That difference can be pretty small when most of the web is also consumed by low effort automatically generated slop. You can't escape it. I know there is a good web out there, but the search engines refuse to give it to me. Probably their algorithm. preferring recent content over good content.

I suspect the only real answer is an economic one, Something like Kagi, where hopefully by paying for results you change from being the product to a customer and this is incentive enough for them to provide good results.