[motorest]

> Seems like the LLM is giving correct output if it’s generating a plausible string of tokens in response to your string of tokens.

No. If you prompt it to get a response and then you ask it to cite sources, if it outputs broken links that never existed then it clearly failed to deliver correct output.

[vrighter]

"correct" for an llm means "fits the statistical distributions in the training data"

"correct" for you is "truth that corresponds to the real world"

They are two very different things. The llm's output is, very much, correct. Because it was never meant to mean anything other than similarity of probability distributions.

It's not what you wanted, but that doesn't make it incorrect. You're just under a wrong assumption about what you were asking for. You were asking for something that looks like it could be true. Even if you ask it to not hallucinate, you're just asking it to make it look like it is not hallucinating. Meanwhile you thought you were asking for the actual, real, answer to your question.

[vharuck]

Right, the dialogue between the user and the LLM closely resembles documents used in training the LLM. People argue with, lie to, and misunderstand others on the internet. Here's a totally plausible hypothetical forum discussion:

Person A: I believe X.

Person B: Do you have a source for that?

A: Yes, it was shown by blah blah in the paper yada yada.

B: I don't think that study exists. Share a link?

A: [posts a URL]

B: That's not a real paper. The URL doesn't even work!

A: Works on my machine.

---

I've seen those kind of chats so many times online. Know what I haven't seen very often? When person A says "You're right, I made up that article. Let me look again for a real one, and I might change my opinion depending on what it says."

[soco]

Why isn't the LLM under the wrong assumption? So I don't get from my tool what I need and it's still me at fault? I am not yet ready to bow to the AI overlords, sorry.

[Timwi]

Oh okay, guess all LLMs are just fine then and we don't need to do any further development on them.

[nkrisc]

But are the links plausible text given the training data?

If the purpose is to accurately cite sources, how is it even possible to hallucinate them? Seems like folks are expecting way too much from these tools. They are not intelligent. Useful, perhaps.

[Scarblac]

Seems that's just expecting things that LLMs were not designed for.

It's a token producer based on trained weights, it doesn't use any sources.

Even if it were "fixed" so that it only generates URLs that exist, it's still incorrect because it did not use any sources so those URLs are not sources.

[soco]

Then let's face it: LLMs were not designed to give proper answers. Now that we settled this and the emperor is obviously naked, what?