[bigyikes]

I’ve been starting to wonder, is GPT-3 the beginning of AGI?

I know, I know, it’s just a language model.

But I’ve been thinking. About my thinking. I think in words. I solve problems in words. I communicate results in words. If I had no body, and you could only interact with me via text, would I look that much different than GPT?

Does AGI really need anything more than words? Is it possible that simply adding more parameters to today’s transformer models will yield AGI? It seems increasingly plausible to me.

[p-e-w]

> But I’ve been thinking. About my thinking. I think in words.

The idea that words and thinking are essentially the same (linguistic determinism) was discarded decades ago. Virtually all linguists today agree that while language influences thought, thought operates far beyond the constraints of language, so a "language model" cannot realistically hope to reproduce the entire gamut of human thinking.

[dllthomas]

> so a "language model" cannot realistically hope to reproduce the entire gamut of human thinking

That assumes that a "language model" actually restricts itself to "language" as the term is used by the linguists. I strongly expect the boundaries won't match exactly, although I have no particular hunch (much less strong argument) that they will disagree enough in the right ways to make bigyikes' suspicions correct.

[blueprint]

not to mention, probably the more crucial fact that just because a language model might be capable of reproducing the entire gamut of human thinking, doesn't mean that arbitrary elaborations via that model have any correspondence to what we are referring to as thinking, or consciousness or learning

[bismuthcrystal]

Or perhaps words are the byproduct of the real thing. Consider the moments where your mind just click and solves something. You find it hard to map words to what happened. Or when you judge a situation to be dangerous, you just kind of know it, and then you map your gut feeling into words so you can explain to someone.

Perhaps "intelligence" is the process that enables these leaps between islands of words.

[whatever1]

Words are just an interface we use.

I believe that thinking and idea generation is much more abstract than words. Animals seem to do a lot of idea generation (improvisation) without knowing about words.

But they cannot pass this knowledge efficiently, except from imitation.

[jakeinspace]

If you had no body starting this moment, your mind would still have benefited from years of interacting with and receiving stimuli from the physical world. GPT-3 isn’t so fortunate

[bigyikes]

Those years of training are surely helpful, but are they strictly necessary? I don’t see why AGI couldn’t infer much of the experiences by reading text.

[masswerk]

It's quite literary the embodiment of statistical probabilities in this corpus of texts. So you have to start with a rather massive corpus and the complexities found in this corpus will define the complexities possibly found in the model.

[mxkopy]

I also had this thought. Something I read about semiotics gave me (or spelled out) the idea that the brain communicates to itself using language, however abstract.

https://mcgovern.mit.edu/2019/05/02/ask-the-brain-can-we-thi...

Turns out, that might not be the case-i.e. understanding is probably not a linguistic phenomenon.

Sometimes, after I put down the crack pipe, I think that understanding and experiencing are two names for something that's fundamentally the same. When I'm thinking about code or a proof, my brain filters out the spacing of the letters, the smell of the paper, etc-which are things I'd do while I'm reading it for the first time. There's these common tools/filters - intuitions - that are exactly the same in thinking and experiencing.

I don't think GPT3 can understand color, for example. But if we fed it a bunch of RGB transforms and raw data, and it generalized and applied them perfectly, could we say then that it's generally intelligent? IDFK

[foota]

Imo what's really missing is a kind of consciousness, or something to drive the system. If gpt3 could be adapted to run over some internal state instead of just a chat log, and came with some system that updated it (and that state stayed internally consistent), I'd be more inclined to believe it was closer to general intelligence.

[knaik94]

There's one big weakness in all current language models that I feel holds it back. There's no way proactive way to have it be persuasive.

Weak AGI will be the first language model that is able to somehow influence the thoughts of the person communicating with it, I think that is the milestone of AGI. From my experience with GPT-Neo and OPT and using it to help write stories or make chatbots, the responses are still very reactionary. In that sense, adding more parameters helps the model give a more coherent response, but it's still a response.

[prng2021]

Babies learn a ton of things way before they understand and use even single words. It takes them years to use sentences but they will have learned a shocking amount by then relative to a newborn.

[hoseja]

I don't, really. Especially for things that matter. I think in abstractions, half-formed references, symbols, shapes. Words are cheap knockoffs of those made for mass consumption.

[sn41]

Please see the article itself. One that really resonates with me now is "putting a child to sleep" - that at often is one of the most trying things I do these days.

What often gives a clue that the child has slept is a slight change in muscle tone, or a sigh, perhaps the pace at which the baby sucks its thumb. How can that be translated into words?

[sweetbitter]

You only think in words?