[joegibbs]

In this case the problem is caused by citing sources - Google finds a list of links using their existing search tech and then produces a response using RAG based on the search results.

That’s where the problem is - it’s originally citing a Reddit post where someone recommends it as a joke, then Business Insider through citogenisis from the original story.

Pure LLMs (no RAG) don’t make this mistake - Claude will tell you it’s a bad idea and will taste bad.