| The Tears in Rain monologue occurred to me as well while I was reading the post, but I don't think it's quite the same for one important reason: the replicants have experienced those things and processed them in whatever sense it is, but LLM-style AIs as we have them now are always inferring what those experiences are like. If you had a fully functioning model in some setting, interacting with the environment and then reporting back to you about it, it might be one thing. But telling you what others have said about it is different. Humans do this too, but there's real-life experiences informing it also. An LLM hasn't fell in love, it simply reports what others have said and infers what it is like to be in love. I think too the piece points to another related thing, which is that someone who has actually experienced something firsthand has some knowledge that someone who has not does not. It might take some extensive sampling to find out what that is, but eventually you'll stumble on it. So e.g., the Sistine Chapel example is sort of telling in this way. Sean basically says "everyone has seen pictures of the Sistine Chapel, if you are asked about it you can tell me what it looks like" but then points out that people don't talk about what it smells like, so if you had been there you might remember it. It's a bit of latent or hidden information, kind of like a secret key, but one that might be informative or useful in some unexpected scenario. I think ultimately this is what the stochastic parrot idea is about: it's not just about mimicking speech patterns, it's about regurgitating what is said about X from third party Z, without being able to produce some additional information not available from Z except by inference. There's no original uninferred information. The inferences might be powerful and highly accurate in their predictions, but they are not providing anything fundamentally original from the experience in a memory sense. Maybe that's what it is? LLMs have no firsthand memories, they only have secondhand memories and inference. They're missing information that would be available through firsthand memories, constrained by the scope of sensory channels. Again, I think you could envision models in some system that are essentially replicant-like, but that's not what our current situation is with standard LLMs. |