[awkwardpotato]

Do you check these sources? I find Gemini and, especially, Google Search AI to regularly cite sources that does not say what it claims to says. For example, (not an actual example but along these lines), "Can Google Sheets do x feature" and it replies "Yup" and links to an Excel YouTube tutorial as its source

[Telemakhos]

I ask ChatGPT and Grok questions about Latin and Greek all the time, and they'll brazenly invent sources, quoting them in Greek or Latin. As an example (an actual example), I asked ChatGPT to round up all the poetry that, like Catullus' sparrow and Statius' parrot, dealt with birds. It hallucinated a bird poem by Callimachus that it claimed was the prototype and gave me not only an English translation but a Greek original—that never existed. It just plain lied. I have zero faith in any fact about the ancient world that comes from an LLM.

On the other hand, LLMs do a great job translating between languages, which is probably why they can vibe code. They catch some grammar errors, too, although not all of them, and even some stylistic errors, so it's useful to run Greek compositions through them. Ask it about linguistic questions ("Which Greek verbs other than ἀφίημι violate Grassman's law?"), though, and it will spew a bunch of irrelevant examples that don't pertain, because it doesn't actually understand what it's doing, just predicting tokens.

[gessha]

What doesn’t help the community is that “hallucinate”, “cite sources” still doesn’t capture what the LLM is doing. LLMs were pre-trained to do one thing, trained to do another and maybe fine-tuned for yet another thing. Do they hallucinate? From our perspective they do because we know true and false but from the tool’s perspective, it’s “just interpolating the text crammed inside of it”.

[solid_fuel]

I find the more helpful understanding boils down to "all responses from an LLM are a hallucination, some are useful"

[Mistletoe]

The sources look good on the one I posted to me.