[everdrive]

I think this is actually a tough one. Yes, I agree that an LLM can never experience this. But -- when a real human needs to translate that experience back into words, it can actually do so just fine.

In other words, when a real human needs to translate an experience into words to share it, A LOT of information is lost. The LLM might actually be better-suited for the the task in your example, even if the point you were driving at stands.

[keiferski]

I don't think it's all that difficult. Language is not a 1-1 match to reality. Ergo any entity which is inherently language-only, will fail to accurately describe the experience.

[brazzy]

Not if the only result you can judge is also language-only, and the entity has access to a large corpus of descriptions produced by people who have had the experience.

[keiferski]

Then you're just comparing descriptions, and you're also dependent on the language skills of the experiencers, and you're still stuck with the problem that language is not a complete map of reality. So an LLM may be able to give you the optimal description of a feeling, on par with what a person would give you; but it won't be equivalent to the actual feeling.

[rainprincess]

This is related to a statement in semantics called "the map is not the territory"

[everdrive]

>and you're still stuck with the problem that language is not a complete map of reality.

Definitely agreed, but at least when talking about experience we've been stuck in this position for most of human history. How do I know what it's like to eat a hamburger? Perhaps, like the cute visual in Ratatouille there is a deep inner experience but (also like in Ratatouille) when I need to communicate that experience with another person I'm stuck relying on language, and the language is found lacking.

I know it's just a kid's cartoon, but interestingly Ratatouille was making this exact same point -- he really lacks any meaningful way to convey his inner experience. It's locked behind language. The film tackles this by showing an Remy's complex and interesting visual, but his friend only gets the fuzziest, dullest visual when he attempts to communicate it.

Remy's immediate rat & family peers actually _never_ get to understand his experience, and instead later help him out of love an allegiance.

https://www.youtube.com/watch?v=xizttM_Cbuc

[brazzy]

Does it matter (why and how) if you can't tell the difference, no matter how hard you tailor the question towards that end?

It's basically the Turing test.