[snailmailman]

This isn’t surprising to me at all. These services put too much trust in “the most common answer” when that might not be the correct answer. Just because people think one thing doesn’t make it true. It’s super easy to spread misinformation online. And if you can SEO to the top, the AI will think your site is correct.

I see factually incorrect “ai summaries” in search results all the time and see that it cites ai-generated slop blogposts that SEO-hacked themselves into taking up the entire first page of search results. This is most common for recent stuff where the answer simply isn’t certain but these AI services will assert something random with confidence.

Not even for news stuff specifically, I’ve been searching about a new video game that I’ve been playing and keep getting misleading obviously incorrect information. Detailed, accurate game walkthroughs and wiki pages dont exist yet so the ai will hallucinate anything, and so will the blogspam articles trying to get SEO ad revenue.

[XorNot]

The problem is AI isn't being used to do what it should be good at: consuming a vast amount of data, following logical connections and thus being able to determine the veracity of claims.

AI should be good at finding logical contradictions and grounding statements against a world model based on physics...but that's not how LLMs actually work.

[simianwords]

>This isn’t surprising to me at all. These services put too much trust in “the most common answer” when that might not be the correct answer. Just because people think one thing doesn’t make it true. It’s super easy to spread misinformation online. And if you can SEO to the top, the AI will think your site is correct.

Yeah I want the answer that the world has converged on and not some looney answer.

It seems like you have never used AI (like in ChatGPT or Gemini) to fact check claims. It doesn't care about blogspam or anything and it prioritises good and factual websites.

[snailmailman]

I have used various AI models. They often aren't good at fact checking. They hallucinate mistakes all the time and often even insist that they are correct when they are not. Some models are better at this, but the ones used in "ai summaries" on various search engines aren't in my experience.

I want the answer that is actually correct. I dont want the ai generated "answer" that has the most SEO, or has appeared often in results, but was entirely made up. When i say "the most common answer" can be incorrect - there are often recent topics where reliable sources don't exist yet. As an example, I've been playing through silksong recently, and had some questions about some late-game content. The official wiki is extremely incomplete, as the game is brand new. I had several questions, and asked a few AI's and search engines. I got completely wrong information a few days ago, but trying the the questions again today, some AI's are only now giving me the right answer. But the correct information simply doesn't appear online enough or in the right places.

(To avoid game spoilers - I wanted to know the quantity of an item, because i wanted to know if i found them all. the answer is "at least 4" because i found 4 of this item while playing. Nearly every AI said "3" a few days ago, and many still do.)