[larryfreeman]

I suspect that there is something else going on than intelligence which will become obvious over the next few years.

There was a horse, "Clever Hans" who appeared to have the ability to answer surprisingly complicated mathematical questions. Did "Clever Hans" have mathematical intelligence. Not at all. He was responding to a cue unknowingly being given by his trainer.

I suspect the same thing is happening with ChatGPT. What if all that is happening is that the text is being formulated to very complicated cues that are implicit in the very complicated, statistical analysis?

https://en.wikipedia.org/wiki/Clever_Hans

[gwd]

A month or so ago I was doing some analysis on our mailing list traffic. I had a complex SQL query (involving tables mapping variations of email addresses to names, and then names to companies they worked for within specific date ranges), that I'd last modified a year previously (the last time I was doing the same sort of analysis), and didn't feel like wrapping my head around the SQL again; so I pasted it into GPT-4 and asked it, "Can you modify this query to group all individuals with less than 1% total contributions into a single 'Other' category?" The query it spat out worked out of the box.

Whatever it's doing, at least for code, it's not a glorified Markov chain -- there's some sort of a model in there.

[larryfreeman]

I agree. The model is where the intelligence is which is the compressed intelligence latent in the training data.

I am arguing similar to John Searle that the processing is not intelligent. The model is a Searlean rulebook.

https://en.wikipedia.org/wiki/Chinese_room

[gwd]

I've always disagreed w/ Searle re the Chinese Room. My guess is that Searle never built an adder circuit from logic gates: combining irrational elements together into something rational is the core magic of computer science.

If you want to see someone asking humans questions where they consistently fail to be rational, to the extent that they sometimes seem to approximate a stochastic parrot, read Thinking Fast and Slow by Daniel Kahneman. (It might actually be interesting to give GPT-4 some of the questions in that book, to see how similar or different they are.)

[larryfreeman]

I'm not sure why you disagree with the Chinese Room argument. I would be interested. I agree that Searle was solely a philosopher who did not take an engineering viewpoint.

Searle's main point is that if I have a book that tells me how to respond and I never learn Chinese, then I do not understand Chinese. If you see a flaw in this reasoning, I am very interested.

My point is just that LLM models are a compression of the content available on the internet equivalent to a rule book. It is definitely fascinating how powerful LLMs are as far as summarization and forming coherent responses to input.

I am a big fan of Kahneman and agree with you that it is will be very interesting to ask GPT-4 the questions in that book.

[Dylan16807]

> Searle's main point is that if I have a book that tells me how to respond and I never learn Chinese, then I do not understand Chinese. If you see a flaw in this reasoning, I am very interested.

You don't understand Chinese, but you are not the process. For the process to understand, it doesn't require any single component to understand like some variation on the homunculus.

And while it might seem obvious that the bulk of understanding can't be contained in a book, you don't really have a book in the Chinese room. Not if the room does a competent job. You have some kind of information-dense artifact that encodes an enormous understanding of Chinese in an inert form. A sweeping library that covers uncountable nuances in depth.

Or to phrase it as a direct attack on the argument: The book does have semantics. You don't need qualia to have semantics, especially not the definition of qualia where nobody can prove they exist.

[larryfreeman]

Thanks! I appreciate the explanation. I think that you put your finger on the major assumption of the argument.

[gwd]

> Searle's main point is that if I have a book that tells me how to respond and I never learn Chinese, then I do not understand Chinese. If you see a flaw in this reasoning, I am very interested.

To a degree, I feel like the Chinese Room argument is begging the question. When I imagine Searle sitting in a room, with a book of instructions and paper and everything he needs to execute GPT-4's equivalent, I basically see an actual computer. That is literally what he is; there is no difference. So then to ask, "Does this system understand Chinese?" is literally exactly the same question as "Does GPT-4 understand Chinese?" You haven't actually illuminated the question in any meaningful way, except to give people not familiar with how microprocessors work a better intuitive understanding. (Which, upon reflection, probably is a fairly useful thing to do.)

I looked a bit at the "1990's version" of his argument on the Wikipedia page you quoted. Going back to my earlier example, this is sort of what his argument sounds like to me:

A1) Electronic gates just on and off switches.

A2) Numbers and addition are semantic.

A3) On and off switches are neither constitutive of, nor sufficient for, semantics.

Therefore, computers cannot add; they only simulate the ability to add.

Now I'm not up on the fine details of what "syntactic vs semantic" means in philosophy, so maybe #2 is't accurate. But in a sense it doesn't matter, because that communicates how I feel about Searle's argument: "I've made some distinction between two classes of things that you don't understand; I've defined one to be on one side, and the other to be on the other side; and therefore computers can't understand."

My best guess as to the "syntactic / semantic" thing is this: In some sense, even his premise, that "Progams are syntactic", isn't actually accurate: Computers operate on bits which are operated on by gates: gates and bits themselves don't inherently have symbols; the symbols are an abstraction on the bits. Even bits are abstractions on continuous voltages; and voltages are ultimately abstractions on quantum probabilities.

What a given set of voltages "means" -- whether they're numbers to be added, or words to be word-checked, or instructions to be executed, or a JPEG to be decompressed, depends entirely on how they're used. If you jump into the middle of a JPEG, your computer will happily try to execute it, and if you dump the program into your video buffer, you'll get a bunch of strange dots on your screen.

Furthermore, when you build an "adder" out of logic gates, you can build the gates such that they correspond to our intuitive idea of binary addition, with individual carries for each bit and so on. But this is inefficient, because then you have to wait for the carries to cascade all the way through the whole thing you're trying to add. Instead, you can brute-force a set of logic gates such that given these 16 bits in, and these 9 bits out (8 plus overflow), you just get the right answer; this will be a lot faster (in the sense that the signals have to go through fewer gates before stabilizing on the final answer), but the gates inside then don't make any sense -- they're almost a "compression" of the longer, carry-based method.

Does that mean that an adder made this way isn't "actually" adding? In the end it doesn't really matter: 16 bits come in, and 9 bits come out the way you want them to. It doesn't really matter that much what happened in the middle.

Putting all that together: It seems to me the "semantics" of a set of bits is based on how they end up interacting with the real world. If I can ask GPT-4 what's missing in my pancake recipe, and it can tell me "you're missing a leavening agent like baking powder", then it seems to me there must be semantic content in there somewhere, and all the arguments about syntax not being sufficient for semantic turn out to have been proven wrong by experiment.

[H8crilA]

Clever Hans is just a proxy. ChatGPT and other LLMs obviously can process information on their own. These two have nothing in common, even GPT-3 would have noticed this.

[larryfreeman]

Let us disagree on what is "obvious". Given an input and an output, you believe that the complexity of the output proves that intelligence takes place.

I agree that ChatGPT is more than a proxy. Unlike Clever Hans, it is processing the content of the question asked. But it is like Clever Hans in that the query is processed by looking for a signal in the content of the data used to train ChatGPT.

The real question is where this intelligent behavior comes from? Why does statistical processing lead to these insights?

I believe that the processing is not intelligent primarily because I see that holes in the data available leads to holes in reasoning. The processing is only as good as the dynamics of the content that it being processed. This is the part that I believe will become obvious over time.

[H8crilA]

I just said that they process information on their own, and this is indeed obvious - you can download and run LLaMA on an airgapped machine.

[larryfreeman]

Agreed. LLMs process information on their own.

I thought you were saying it was "obvious" that the processing demonstrated intelligence.

My point was the level of intelligence shown is relative the quality and quantity of the data used for training. The data is where the intelligence is and the model is a compression of that latent intelligence.