|
|
|
|
|
|
by widdershins
521 days ago
|
|
|
> it does so with no consideration of worldly facts Why don't you consider its training set (usually the entire internet, basically) worldly facts? It's true that the training set can contain contradictory facts, but usually an LLM can recognize these contradictions and provide analysis of the different viewpoints. I don't see how this is much different from what humans can do with documents. The difference is that humans can do their own experiments and observations in the real world to verify or dismiss things they read. Providing an LLM with tools can, in a limited way, allow an LLM to do the same. Ultimately its knowledge is limited by its training set and the 'external' observations it can make, but this is true of all agents, no? |
|
|
But at inference time it’s not referring to that data at all. Some of the data is aliased and encoded in the model’s weights, but we’re not sure exactly what’s encoded.
It may very well be that vague concepts (like man, woman, animal, unhealthy) are encoded, but not details themselves.
Further, at inference time, there is no kind of “referencing” step. We’ve just seen that they can sometimes repeat text they were trained on, but sometimes they just don’t.
The LLM based systems you’re probably using do some RAG work to insert relevant information in the LLM’s context. This context still is not being referred to per se. An LLM might have a document that says the sky is red, but still insist that it’s blue (or vice versa)
So while the info an LLM may have available is limited by its training data and the RAG system around it, none of that is guaranteed at inference time.
There’s always a significant chance for the LLM to make up bullshit.