[photochemsyn]

> “It’s like imagining that a printer could actually feel pain because it can print bumper stickers with the words ‘Baby don’t hurt me’ on them."

Problem: the human brain has no pain receptors, no nociceptors. It just takes in messages from remote nerves and 'prints a bumper sticker' that tells higher cognitive centers 'you're feeling pain!'. What's the difference?

> "LLMs are like a search engine that rephrases information instead of giving it verbatim or pointing you to the original source."

Problem: How does this differ from human learning? If a human reads a book and tells someone else about it, constructs a summary of the important points and memorable passages, how is that fundamentally different from what LLMs are doing?

The second one really impacts the intellectual property arguments - if training a model on data is fundamentally similar to training a human on data, does 'derivate work' really apply to the creations of the human or of the model?

[Barrin92]

>What's the difference?

The pain receptors. The human brain doesn't just "have" pain receptors. Your entire body, including your brain, is one system. Your brain isn't piloting your body like a mech. This brain body dualism is a misconception of how biological organisms work. You are your pain receptors just like you are your brain, and removing any part would alter your perception of the world.

>How does this differ from human learning?

It differs from human beings in every respect. Humans don't do linear algebra in their head, biochemical systems are much too slow for that. Humans don't inhabit some static model of the world learned at some fixed point t, you're a living being. Your brain wasn't trained four months ago and was done at that point. Humans learn with a fraction of the information and through self play, they don't decohere, and so on.

[photochemsyn]

If you were technologically advanced to the point of being able to float a human brain in a vat, and connect up all the nerves going into that brain to synthetic sensors with the appropriate signalling outputs, then you could construct a synthetic reality for that brain that it would not be able to distinguish from its previous brain-in-a-body experiences.

As far as learning, human learning is certainly much slower than machine learning but it's not really clear at a biochemical-molecular level that they're entirely different, eg the formation of memories and so on, considering a wide range of alternate hypothesis before selecting one, etc.

[Barrin92]

>you could construct a synthetic reality for that brain that it would not be able to distinguish from its previous brain-in-a-body experiences.

No. I'd recommend reading Dennett's Consciousness Explained for a longer treatment of this, but if you want to have an experience just like you, you need a body and an environment just like you. Sure it could be synthetic in the sense of it being artificially constructed, you could have artificial limbs, but it can be no different from the one you have, it cannot be a vat. There is no "Cartesian Theater" in the brain, your experience is already distributed throughout both your brain and body. Your experience isn't something being "fed into you" (the brain) from the outside, the entire perception emerges in the first place through being embodied in the world. The concept of the thought experiment itself would not even make sense without implicitly assuming an embodied reality beforehand setting it up.

Just like there is no philosophical zombie that's somehow materially identical but without sentience, the reverse doesn't exist either. There is no disembodied computer with the experiences of an organic being because they function in entirely different ways.

[paulryanrogers]

Humans are far more self contained than most LLMs which depend upon a lot of electricity and possibly an array of loosely connected components. LLMs also don't really have the signal bandwidth of human nervous systems yet. They're capacity to interact with the physical world is also seriously limited for now.