1.3% isn't great. I'd rather just go, and pay, directly to trusted news sources. Everyone has different tolerance for falsehoods and priorities I guess.
As others have already pointed out, feeding these new articles aren't magically going to make them any more accurate. These hallucinations are going to be on top of any errors in the data sources.
I'm not replying to point that out, I think others have done a better job. It's mostly that this conversation made me think of this classic Babbage quote that I've always enjoyed.
"On two occasions I have been asked, – 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question"
Except when that happens, a clarification is almost always added at the bottom of the article ("This article was amended on [date]. An earlier version said xxx" or some variation thereof).
You're not gonna get a second push notification from an AI summary saying "Oopsies, the previous notification was wrong". Once it's out, it's out, and that sort of damage is difficult to repair.
Yes, but that's going to be on top of the ~1.3% hallucination rate (largely, there's always some very small chance it hallucinates the truth when the article had it wrong - but basically not worth considering).
Anything other than 0% is borderline immoral. Imagine sending a push notification to somebody's phone with a completely made-up headline summary. Even if it happens once in a hundred times, that's too much.
Things like that slowly but surely erode trust and make it harder and harder to trust anything that's generated by AI, especially when it comes to news, where trustworthiness is essential, and probably the main reason people pay for news. See for example https://www.bbc.co.uk/news/articles/cge93de21n0o
This is a ridiculous standard. News headlines at the moment would have an error rate wildly above 1.3%. The articles about Apple having trouble with LLM headlines is that the on-device model is weak and it's trying to compress too much into too few characters. I'd guess the chance of Gemini incorrectly summarising an article to be almost 0%.
Have you ever read a news article on a subject where you have expertise and knew it was inaccurate? The news is probably more inaccurate than you think.
I bet you think the news is accurate all other times. It’s called “Gell-Mann Amnesia”
You’d have to pay quite a bit to get journalists to answer your questions specifically.
The whole isn’t about generating news articles, it’s about getting the model up to date on facts so it can synthesize a newspaper for you. I’d say it’s a way to get journalists to be journalists again instead of clickbait composers - as long as the model doesn’t inject clickbait there itself. I don’t trust Google to not do it sometime, but they aren’t doing it now and the infrastructure is being made for others to consume when Gemini suffers from inevitable enshittification.
> You’d have to pay quite a bit to get journalists to answer your questions specifically.
This isn't what I meant. I pay directly for subscriptions/donations to news organizations that employee journalists that do this original reporting. I don't want a middle man that just messes it up. This goes for LLMs and for free news sites that don't do much more than summarize original reporting. I've seen more than a few times where they inject opinions, mess up facts or put focus on what was originally a small side point in the article.