[swatcoder]

> "a different insight"

What a generous euphemism. Depending on the topic, it's not unusual to see Google AI search referencing links that indicate the exact opposite of what it presents in its prose.

So many of these tools are presented as if they're doing something you'd intuitively expect of a human operator, like collating search results and then summarizing them in this case, but the actual operation of them is so alien to us that our intuitions don't apply and these presentations are all but fraudulent.

Because it's presentation cites references, and because those references sometimes reinforce the expression of the AI prose, we grant it trust, but it doesn't use those references in the way a human would and so the relevance of them being cited is radically different (and weaker) than we're accustomed to.

The closest example in human behavior might be the rushed, naive student who just pastes citations into an already written paper at the last minute, but even that's an anthropomorphism that misrepresents the alien ways in which the tool works.

[jtbayly]

Yesterday ChatGPT gave me 5 references to a site that doesn’t exist. It’s such a bummer how useless these tools are so much of the time, while seeming so helpful. In fact they are actually helpful, which is why it’s so irritating.

[echelon]

Meanwhile, I asked ChatGPT to verify the ALIAS DNS records of its search results so I could build evidence Netlify was blocking me [1], and it worked flawlessly.

I've replaced Google almost wholesale with ChatGPT and Claude.

[1] If you use Claude Code to assess a Netlify website (even your own), they'll block your IP address from the entire netlify.app IP range.