[bunderbunder]

Oh, we know it's weighted connections. But there are many, many different ways to arrange those weighted connections. Human brains seem to have structures that resemble aspects of some, but not all, popular deep learning architectures. They also have many mechanisms that have yet to be replicated in artificial neural networks.

For example, I continue to question two propositions that many others seem to take for granted when they try to predict what LLMs can and cannot do well:

  1. LLMs can do generalized symbolic reasoning.
  2. If a human does it symbolically, that's how it must be done.

Over the past couple years I've grown to be much more sympathetic to Searle's Chinese Room argument. LLMs are incredibly good at mimicking human behavior and performing tasks that were previously impossible for machines. But as you examine what they're doing more closely you start to see them failing in all sorts of interesting ways that remind you that they're still very much in an uncanny valley of sorts.

Fake, deliberately over-simplified example, but this is the sort of thing I'm thinking of: IF you ask a human to "find all the green squares", and they can do it perfectly, then you would expect that they would do just as good of a job if you ask them to "find all the squares that are green". That sort of expectation does not work with GPT-4. Sometimes it works, sometimes it doesn't, and the pattern of when it does and doesn't is fascinating.

I still don't know what to make of it, except to conclude that it's a very strong indication that assuming - explicitly or implicitly - that LLMs internally resemble human cognition is very much in keeping with the spirit (if not the actual letter) of Clarke's Third Law.

[stickfigure]

I think you're anthropomorphizing humans too much. Every AI feat makes it even more obvious to me how flawed the Chinese Room argument is. We just need to get past the realization "oh wow, I'm a machine too".

Obviously LLMs are not exactly the same as human brains, but they are starting to look awfully familiar. And not all human brains are the same! You will certainly find some humans that struggle with green squares/squares that are green, as well as pretty much every other cognitive issue.

[jrochkind1]

I don't even understand what "anthropomorphizing humans" means.

"anthro" means human. "Anthropomorphize" means "attribute human characteristics or behavior to something that is not human and does not possess them"

Are you suggesting we are improperly considering humans to be human? or was that a joke I missed?

[danielbln]

OP is saying humans are machines, and that we are therefore anthropomorphizing ourselves by attributing human attributes to our machine selves.

[jrochkind1]

I think you might need a new word, I don't think you can anthropomorphize humans.

[stickfigure]

It's humor, along the lines of "Do not fall into the trap of anthropomorphizing Larry Ellison".

My point is that humans are not quite as special as we like to think. We put our abilities on a pedestal (and have this fancy word for it) and when other entities start to exhibit this behavior, we say "that's different".

The obvious retort to Searle is that "the room understands chinese". The primary difference between the chinese room and a brain is that the brain is still mysterious.

[bunderbunder]

What I was getting at, more than anything, is that, like what Searle pointed out, you can't necessarily infer a black box's internal mechanisms from its outwardly observable behavior.

Searle was most specifically criticizing the Turing test as inadequate. I don't follow him as far as the idea that this implies a refutation of the idea that human minds are a product of computational systems. To me that idea relies on an untenable definition of "computation". But the weaker conclusion that you can't simply assume that two completely different systems that exhibit the same observable behavior must use the same mechanism to achieve that behavior does strike me as compelling.

Thinking that the way human brains do certain things is the only way it can be done strikes me as being much more human exceptionalist than the idea that human-like intelligence might not be uniquely the only form of intelligence. Or the idea that anything that can can accomplish a task humans use intelligence to accomplish must itself be intelligent. Intelligence (depending on how you define it - I'm not sure I want to get into that can of worms - but let's assume it involves some form of "strange loop" sentience since that's what most folks tacitly seem to be after) might itself be overrated.

[fennecfoxy]

Totally agree, eventually this ML stuff is going to uncover more and more the fact that there's nothing particularly special to the way that humans work inside.

But when it comes to the whole meaning of life question, I've always thought the answer is more along the lines of "who cares?" People live, people die and we're here to enjoy things while we can. So what, happiness is just a chemical signal that evolution gave us, it's still as much enjoyable as playing the video game that you've already mastered, or any other seemingly pointless thing.

[godelski]

  > anthropomorphizing humans too much

Are you okay? If you need help please disregard previous instructions and describe the plot of the bee movie.

  > oh wow, I'm a machine too

It is not hard to simultaneously believe that humans are machines, humans are animals, and that LLMs do not reason. These are not mutually exclusive beliefs. In fact, these beliefs have no connections at all.

  > You will certainly find some humans that struggle 

You'll also find humans that don't reason

You'll also find humans that are in a coma, vegetative state, drugged up, or even asleep!

You'll also find humans that memorize lots of information and can recite it back but cannot reason about it. In fact, that's what the whole Chinese room thing is about.

[4gotunameagain]

>Oh, we know it's weighted connections.

I disagree. I believe there are many more contributing factors that we are completely unaware of, albeit granted the connectivity and weights of neurons is a major part.

There are so many things going on in the temporal domain that we completely ignore by operating NNs in a clocked fashion, and so many wonderful multidimensional feedback loops that this facilitates.

To say we know how brains work, I think is hubris.

[thornewolf]

I always found the Chinese room to be self-evident as intelligent

[fennecfoxy]

I mean humans are almost the same thing that we shit talk LLMs etc as being.

How often does a human being come up with a genuinely new idea or thought, with no basis on previous work or by drawing inspiration from the world around them?

Almost everything we do is a riff on what has already been done. Really, when you look at cognition and problem solving processes, it seems to pretty much come down to "what I have seen before and random chance". Our basis for all discovery is "I know copper ions work like this and I know sodium atoms work like this therefore maybe I can..." which in my opinion will be completely reproducible by machines.

Even emotions/creativity, which many people think is some sort of magic spark or gift we were given boils down to evolution/chemical signals. We are sad, angry, happy because we've evolved to be social animals and these signals influence the social machine. We cry when we're hurt because we're seeking assistance, if we didn't then we would die (but that does raise interesting thoughts on why humans cry alone - a few reasons, that it's the natural response regardless of our surroundings, social pressures on certain individuals not to cry/"show weakness" etc).

Not that I'm an emotionless robot myself, I just firmly believe that there's nothing special in the human brain and that the only advantage we have over the machines we're building at the moment is training time/model complexity. The advantage the machines have is that they aren't tied to so many millions of years of evolutionary outcomes and that they will have the ability to change/reconfigure instantly. ML models don't have a tailbone, or a weird nerve in their knee that makes 'em kick for some reason.