[codebje]

Because LLMs aren't sentient, they don't draw on facts, and they don't have nuance. The answer given is similar to answers you might expect to see for similar questions.

It's really amazing we can make machines do that, and it's really depressing that we think a stochastic bullshit machine is going to give us something we can rely on.

[Robotbeat]

Or… the default LLM Google uses for search has been quantized to s**. Ask a proper Thinking model, with browsing enabled, and odds of a correct answer are much higher. There’s been substantial improvement in AI in even the last year.

Ask a human a question like this, and they also have a chance of getting it wrong, even when confident.

[nvme0n1p1]

> Ask a human a question like this

Why would a human know specs for a random phone off the top of their head? The human response is either "I don't know" or "let me look that up", not a hallucination.

[codebje]

I think that it feels a little wasteful to go to Google search to ask a question like this, only for the AI that's giving you an answer instead of page results to perform its own web search to get you the response.

Also, I asked a thinking model with browsing enabled and got this:

> The Google Pixel 10 is expected to support Wi-Fi 7 (802.11be), based on the Qualcomm Snapdragon 8 Gen 4 / Tensor G5 chipset it will likely use, which includes an integrated Wi-Fi 7 modem. Specific finalized specs aren't confirmed until Google's official announcement.

(Model GLM-5-Turbo - two months old - using Kilo Code in the "Ask" profile; in its thinking token churn it reasoned that it should keep the response brief and direct. Perhaps not the best suite of model+harness for this task, but it's what I had to hand that's not quantized to shit, is a thinking model, and has a web search tool available to it.)

[lelanthran]

> Ask a human a question like this, and they also have a chance of getting it wrong, even when confident.

We google something specifically because the humans within reach don't know. The goal of searching is, well, to search pages - we're trying to find a site when we use google search.

The goal when using an LLM is generally different; we want an answer, not a site.

[Robotbeat]

LLMs are not a site. They are a clever person that can point you to sites. They, like humans, are fallible.

[tremon]

LLMs can not point you to sites, only in a general direction. That is because complete URLs do not exist as single tokens in any of the large models. It can synthesize a plausible-looking url, and if you're lucky that URL might even exist. But that doesn't mean that there is any relation between between the text surrounding a hyperlink in LLM output and the text on the linked page.

AI agents can verify and summarize URLs, but a plain LLM can not.

[Robotbeat]

I bow to your correction. I was using LLMs as a sloppy shorthand for modern AI agents with best interfaces.

[jazzyjackson]

*so long as an accurate answer exists on the internet

Claude is OK at saying when it can’t find good information, but it’s still 50/50 on citing a source that has nothing to do with its claim.

[andrekandre]

its bad in dev as well... i've seen llm code review bots tell me things that are flat-out not true; this like "this wont compile because windows 11 doesn't exist" like wtf am i paying for this again?