I was a bit wary of trusting the AI summaries Google has been including in search results… but after a few checks it seems like it’s not crap at all, it’s pretty good!
I think their point is that all of the content out there is turning in to AI Slop so it won't matter if search changes because the results themselves have already been changed.
I have systemic concerns with how Google is changing roles from "knowledge bridging" to "knowledge translating", but in terms of information: I find it very useful.
Always check the sources. I've personally found it;
- Using a source to claim the opposite of what the source says.
- Point to irrelevant sources.
- Use a very untrustworthy source.
- Give our sources that do not have anything to do with what it says.
- Make up additional things like any other LLM without source or internet search capability, despite reading sources.
I've specifically found Gemeni (the one Google puts at the top of searches) is hallucination-prone, and I've had far better results with other agents with search capability.
So... presenting a false or made-up answer to a person searching the web on a topic they don't understand... I'd really like to see a massive lawsuit cooked up about this when someone inevitably burns their house down or loses their life.
I’ve had to report AI summaries to Google several times for telling me restaurant items don’t contain ingredients I'm allergic to, when the cited “source” allergen menu says otherwise. They’re gonna kill someone.