[zanek]

I completely agree with Marcus' assessment of GPT-2 and its ilk. They are simply regurgitating words with zero understanding of any words/meaning.

It seems that OpenAi and others are peddling this AI when its simply a glorified Eliza on steroids.

[Rioghasarig]

I can't agree with you here. While GPT-2 isn't good at filling in particular details (like the language someone from Boston should speak) it is astonishingly good at recognizing the _kind_ of answer that should be produced. The fact that it usually answers with a language here is a reflection of what it understands. And it behaves similarly for a whole range of different tasks. If you write a sentence that should obviously end with the name of a person it will give you the name of a person. If you write something that should be completed by the name of an event it will give you the name of an event. If you write text that should obviously be finished by the description of a person's appearence it will give you a description of a person's appearence. GPT-2's grasp of the appropriate context of a wide range of situations is nothing short of incredible. Dismissing this as "Eliza on steroids" is just short-sighted.

[drongoking]

> it is astonishingly good at recognizing the _kind_ of answer that should be produced. The fact that it usually answers with a language here is a reflection of what it understands. And it behaves similarly for a whole range of different tasks. If you write a sentence that should obviously end with the name of a person it will give you the name of a person.

I think you're anthropomorphizing a fair amount. GPT-2 has memorized huge amounts of text and can, yes, generalize the characteristics of words that fill certain slots. To say that it recognizes the kind of answer that should be produced is implying agency where there is none.

The original author is right. GPT-2 has no idea what it's talking about. Play with it any amount of time and you'll realize this. It's more than "Eliza on steroids," but the impressiveness of GPT-2 comes from the style of its language, not the substance.

[missosoup]

> I completely agree with Marcus' assessment of GPT-2 and its ilk. They are simply regurgitating words with zero understanding of any words/meaning.

There's a pretty strong argument that most humans also frequently do this.

My go-to example is high school physics. The majority of students merely learns to associate keywords in problem statements with a table of equations and a mapping of what numbers to substitute for what variables in those equations. Only a small handful of students actually understand what those equations represent and have the ability to generalise them beyond the course material.

[darkkindness]

Just to support your argument further, here is a related snippet from another comment[0] by knzhou:

> Students can all recite Newton's third law, but immediately afterward claim that when a truck hits a car, the truck exerts a bigger force. They know the law for the gravitational force, but can't explain what kept astronauts from falling off the moon, since "there's no gravity in space". Another common claim is that a table exerts no force on something sitting on it -- instead of "exerting a force" it's just "getting in the way".

Here is some food for thought for educators. If GPT-2 also makes sense of the world by regurgitating what it sees, perhaps this is simply the nature of learning by example, and we should accommodate for this. Perhaps it isn't so effective to give students mounds of problem sets offering clear premises and easy-to-grade answers. Unless you want your students to be GPT-2s.

[0]: https://news.ycombinator.com/item?id=21729619

[thrwaway69]

I wonder if gpt2 or similar projects can be used to make systems to train teachers. Teacher explains something and raise questions or statement and have GPT2 complete them. That way, they can learn more about students, common questions, misunderstandings, etc.

If someone knows more about what companies or tech is used for training teachers, do let me know. I am pretty interested in any vacuum in the industry and if schools pay enough for training their teachers.

[drongoking]

Arguing that physics students don't understand equations very well is a poor way to make a point about GPT-2. GPT-2 fails at a much more basic level, and that's Marcus's point.

Talk to a five year old for a while. The five-year-old's language may be crude but it shows basic concepts of a conversation, continuity, referents, basic causality, etc. GPT-2 has none of these. It regurgitates smooth language fragments because that's what it was trained on, but it exhibits no awareness that it's involved in a communication event with another being. Except possibly at cocktail parties, humans don't simply regurgitate words.

[missosoup]

Physics students generally not understanding physics isn't an isolated case. The point is that the majority of people don't really 'understand' the concepts that they talk about a lot of the time. What does it even mean to understand something? Is it binary? Is it a continuous scale?

There's no test for 'true' understanding, there's no test for awareness. There's fundamentally no way to distinguish a p-zombie from a conscious being. It's possible that p-zombies are also fundamentally incapable of distinguishing themselves from 'truly' aware beings, in which case the distinction between awareness and non-awareness is meaningless.

Trying to detect 'awareness' or 'understanding' in communications is a dead end. There is no reason to believe that a human doesn't also 'just 'regurgitate smooth language fragments because that's what it was trained on'. In fact, I see a lot of that in the professional services world. People have built entire careers by stringing plausible-sounding sentences together even though they're completely devoid of meaning if you actually try to parse them. The most interesting thing is that those people genuinely believe that they know what they're talking about. They rarely admit or believe that they lack understanding, even if it's clear to everyone around them.

[wutbrodo]

I don't think you can handwave away what "true understanding" means, and whether or not humans would meet the bar that you are implicitly setting. I don't think GPT is AGI either, but nobody is claiming it is. The claim is that it's a step along that path, making much more sophisticated and less articulable connections between expressed concepts than was previously possible. It's not clear to me that this is an unreasonable claim.

[russdill]

It certainly understands the connections between both words and the ideas they represent. I'm not sure if that meets your definition of understanding. For instance, if I give it a paragraph or two reading story about a miner, where the first name and last name are mentioned, but not given together, I can ask it to complete: "The miner's full name was" and it will fill in both the first and last names.

[laichzeit0]

Define the word “understanding” in a mathematically rigorous way. Otherwise it’s not clear exactly what you’re even saying in that sentence.

[starpilot]

"Hello"

Was this generated by a human or a computer?

[ackbar03]

Definitely computer. Someone should ban your account