[nonsensikal]

> In a particularly influential article, Emily Bender, Timnit Gebru, and colleagues ...

This quote was included to virtue signal. It assured the article would be garbage.

The thesis (buried a dozen paragraphs in) is

> I suggest much of what large pre-trained models do is a form of artificial mimicry. > Here’s the thing about mimicry: It need not involve intelligence, or even agency. > a kind of biological mimicry that can be seen as solving a matching problem > Artificial mimicry in large pre-trained models also solves a matching problem > We also need a better understanding of the mechanisms that underlie the performance of large pre-trained models to show what may lie beyond artificial mimicry.

Wait what? The whole thesis is that AIs are parrots. But parrots are living intelligence creatures.....

But the entire argument is smuggled in on the assumption

> Parrots repeat phrases without understanding what they mean

This is really simple and boring.

Parrots, "parrot". Parroting isn't understanding. AIs, "parrot". therefore AIs dont understand

The novel contribution is zero, and all this has already been refuted and death marched.

AGI is inevitable because computation is universal and intelligence is substrate independent.

[fzingle]

> AGI is inevitable because computation is universal and intelligence is substrate independent.

What the article is asking, albeit obliquely is: how do we know that to be true?

It is very difficult to prove when definitions for intelligence and consciousness are fuzzy and not widely agreed upon.

[nh23423fefe]

I see people assert this all the time. Intelligence is the ability to achieve goals. Consciousness is contents of what you are aware of.

The definitions are irrelevant. People want a definition to do demarcation. But demarcation is boring. I don't care whether some threshold entity is on this or that side of the line. The existence of some distinct territories is enough for me.

There is no reason to imagine AIs are prohibited from occupying territory on both sides of the line.

AIs aren't climbing up a ladder, non local exploration is possible

[visarga]

> Intelligence is the ability to achieve goals

A more precise definition by François Chollet: The intelligence of a system is a measure of its skill-acquisition efficiency over a scope of tasks, with respect to priors, experience, and generalization difficulty

https://arxiv.org/pdf/1911.01547.pdf

So a system is more intelligent if it can solve harder tasks with fewer trials and prior knowledge. Intelligence is always defined over a scope of tasks, for example human intelligence only applies to the space of tasks and domains that fit within the human experience. Our intelligence does not have extreme generalization even though we have broad generalization (defined as adaptation to unknown unknowns across a broad category of related tasks).

[tbrownaw]

I read it more as asking about deep-NN models specifically.

If it really was trying to suggest that computation isn't universal or that our intelligence is non-physical or something, that would be a whole different problem.

[astrange]

> AGI is inevitable because computation is universal and intelligence is substrate independent.

People who say this slide invisibly from "computers exist" to "you could simulate a human in a computer" to "AGIs exist and have all the properties granted to them in science fiction stories about AGIs". Mostly they do this by forgetting if they have a specific or vague definition of the word "intelligence".

Can it imagine how a human would cook an egg if you only fed it natural language text about doing that? (No, language doesn't describe muscle memory.) Could it cook an egg if it was controlling a robot? (Not without practice.) Would it actually do that if you told it to? (Maybe. Who says it has motivation? All we know is it has intelligence.)

[JieJie]

So, the theories that I've been reading present the picture that what AGIs have already has a sort of consciousness that isn't anything like human consciousness, because these agents really only have a subset of the processing that humans have, which in most cases these days, is language.

Human brains develop by performing movement, in fact, much of our brain is specialized for performing movement, judging angles and speeds, testing if we can jump across a puddle safely. The AGI would need to be trained on these things and the language associated with them. Your example with the egg, I think, is illustrative of the amount of training the AGI would need to go in order to learn, and that training would become part of its "personality".

There is also a set of desires that humans have, such as survival, companionship, hunger, etc. These would also have to be trained. These desires would have to be trained into the AGI as well.

The theory is that if we would have to train an AGI the same way we do humans in order to get it to be close to human consciousness. That doesn't mean it wouldn't have a type of consciousness along the way, maybe closer to what we experienced as we grew from children to adulthood.

Anyway, just conclusions from what I've been reading.

[ffhhj]

> AGI is inevitable because computation is universal and intelligence is substrate independent.

Let's say Human Intelligence and AGI are subsets of General Intelligence, we don't know whether GI will actually subsist without the biological drive, AGI's might decide that hoarding resources/knowledge is futile in a finite universe and shut themselves down then reaching the halt state of this paradox. It might never be possible to achieve an AGI too different from the stealing/warmaking/miraculous HI.

[nh23423fefe]

I dont see how this is anything except lack of imagination. To imagine that life can only be made of meat is pretty strange. To imagine the only way of "being" in this universe, is to be nearly identical in construction and behavior to humans is absurd.

[visarga]

> Parrots, "parrot". Parroting isn't understanding. AIs, "parrot". therefore AIs dont understand

This doesn't sit well with me. Parrots can repeat sounds. AI models can compose phrases into coherent output. Very different. One is just funny and the other can be really useful.

I think the Stochastic Parrots name is an attempt to disparage and limit LMs by a group of people who only see them as discrimination amplifiers.

[tbrownaw]

> AGI is inevitable because computation is universal and intelligence is substrate independent.

That says nothing about timescale. There's no guarantee that the current favored approach will work. Or even that we'll figure it out before we go extinct for whatever far-future reason.

[visarga]

The path to AGI is shrouded in mist and we don't know which way to take. If you directly optimise towards an objective you might not reach it unless you're already very close to see the last few steps.

The best solution is to try everything, build the necessary stepping stones. Some of them will be useful in hindsight, we just don't know which. Biological evolution does the same. Culture evolves in a similar way.

For example, did Newton have any idea about the utility gradients will have in learning models of language? Probably not. But his contribution paved the way to GPT-3. From his position it was just impossible to foresee, from our position it seems trivial. You can learn the basics of transformer neural nets in a course of 8h on YT.

So we don't need guarantees if we are ready to try out everything. And that's exactly what the deluge of AI papers does.

[balsam]

nautilus is an organ of the mediocracy and sadly, the title is more valuable than the article.