[anyonecancode]

Interesting post. I find myself moving away from the sort of "compare/contrast with humans" mode and more "let's figure out exactly what this machine _is_" way of thinking.

If we look back at the history of mechanical machines, we see a lot of the same kind of debates happening there that we do around AI today -- comparing them to the abilities of humans or animals, arguing that "sure, this machine can do X, but humans can do Y better..." But over time, we've generally stopped doing that as we've gotten used to mechanical machines. I don't know that I've ever heard anyone compare a wheel to leg, for instance, even though both "do" the same thing, because at this point we take wheels for granted. Wheels are much more efficient at transporting objects across a surface in some circumstances, but no one's going around saying "yeah, but they will never be able to climb stairs as well" because, well, at this point we recognize that's not an actual argument we need to have. We know what wheels do and don't.

These AI machine are a fairly novel type of machine, so we don't yet really understand what arguments make sense to have and which ones are unnecessary. But I like these posts that get more into exactly what an LLM _is_, as I find them helpful in understanding better exactly what kind of machine an LLM is. They're not "intelligent" any more than any other machine is (and historically, people have sometimes ascribed intelligence, even sentience, to simple mechanical machines), but that's not so important. Exactly what we'll end up doing with these machines will be very interesting.

[sgt101]

I think ChatGPT is a Chinese Room (as per John Searle's famous description). The problem with this is that ChatGPT has no idea what it is saying and doesn't/can't understand when it is wrong or uncertain (or even when it is right or certain).

I believe that this is dangerous in many valuable applications, and will mean that the current generation of LLM's will be more limited in value that some people believe. I think this is quite similar to the problems that self driving cars have; we can make good ones for sure, but they are not good enough or predictable enough to be trusted and used without significant constraints.

My worry is that LLM's will get used inappropriately and will hurt lots of people, I wonder if there is a way to stop this?

[pyrolistical]

We stop this like any other issue with the law. Somebody is going to use LLM and cause harm. They will then get sued and people will have to reconsider the risk of using LLM.

It’s just a tool

[pessimizer]

Or they will be sued and it will be dismissed, or the entire process might be suppressed based on trade secrets, national security concerns, or lobbyist concerns. Then people will have to evaluate the risk of using LLM by making sure they have enough lawyers, enough cash, or enough connections as a contractor to get away with doing whatever benefits them most.

I don't know what just a tool is supposed to mean. Pistols and nuclear warheads are also tools, they're tools for killing people.

[actionfromafar]

Not just for killing, bur for area denial, never forget.

[phonescreen_man]

As in denial of access to an area? Or something else?

[andrepd]

It is a relevant argument though, as a reply to claims of "GPT4 will replace doctors and lawyers and programmers in 6 months".

[aquafox]

Machines don't huse (human) language to communicate with each other, humans do, so comparing LLMs to humans makes sense I think.

[abarker]

Conversely, these models open up philosophical questions of "exactly what a human is" beyond language abilities. How much of what we think, do, and perceive comes from the use of language?

[visarga]

I think most intelligence is in the language. We're just carriers, but it doesn't come from us and doesn't end with us. We may be lucky to add one or two original ideas on top. What would a human be without language?

Language models feed from the same source. They carry as much claim to intelligence, it's the same intelligence. What makes language models inferior today is the lack of access to feedback signals. They are not embodied, embedded and enacted in the environment (the 4 E's). They don't even have a code execution engine to iterate on bugs. But they could have.

And when a models does have access to massive experimentation, search and can learn from its outcomes, like AlphaGo, then it can beat us at our own game. Trained just in self-play mode, learning from verifying outcomes, was enough to surpass two thousand years of history, all of our players put together.

I think future code generation models will surpass human level based on massive problem solving experience, and most of it will be generated by its previous version. A human could not experience as much in a lifetime.

This is the second source of intelligence - experience. For language models it only costs money to generate, it's not a matter of getting more human data. So the path is wide open now. Who has the money to crank out millions of questions, problems and tasks + their solutions?

[bilsbie]

> I think most intelligence is in the language. We're just carriers

This is such a profound idea. I’ve been wondering about that for a while. Is there anywhere to read up on it?

[mike_hearn]

Isn't that just a rephrasing of the Sapir-Whorf hypothesis? If so then it's old and thoroughly debunked. Language features don't seem to influence the way we think, which is another way of saying that intelligence and language are different things. If you want to read about it you can look at the history of this idea in the 20th century from when it was proposed by linguists in the 1930s all the way up to the time it became discredited, as there are many research papers and even books on the topic.

One of the really troublesome problems with Sapir-Whorf and derivatives is that they led directly to some very nasty totalitarian behaviors. In "1984" a core policy of the Big Brother government is Newspeak, in which language changes are (believed to be) used to control the thoughts of the population and establish eternal power for the Party. This wasn't merely a quirky bit of fictional sci-fi, it was directly inspired by the actual beliefs of the hard left. The extent to which Newspeak was an accurate portrayal of life under the Nazis and Communists is explored in "Totalitarian language : Orwell's Newspeak and its Nazi and communist antecedents".

https://searchworks.stanford.edu/view/2016479

Today it's known that Sapir-Whorf isn't supported by the evidence, but there's still a strong desire on the political left to manipulate thought through language. Stanford's recent "Elimination of Harmful Language" initiative is a contemporary example of this intuition in practice. It doesn't work but it sounds so much easier than engaging in debate that people can't let it go.

tl;dr to the extent this has been studied already, intelligence is not in the language.

[bilsbie]

I’m thinking of something different but you raise some good points regardless.

[morelisp]

They don’t “open” shit, the linguistic turn came and largely went long before they existed.