[missosoup]

> I completely agree with Marcus' assessment of GPT-2 and its ilk. They are simply regurgitating words with zero understanding of any words/meaning.

There's a pretty strong argument that most humans also frequently do this.

My go-to example is high school physics. The majority of students merely learns to associate keywords in problem statements with a table of equations and a mapping of what numbers to substitute for what variables in those equations. Only a small handful of students actually understand what those equations represent and have the ability to generalise them beyond the course material.

[darkkindness]

Just to support your argument further, here is a related snippet from another comment[0] by knzhou:

> Students can all recite Newton's third law, but immediately afterward claim that when a truck hits a car, the truck exerts a bigger force. They know the law for the gravitational force, but can't explain what kept astronauts from falling off the moon, since "there's no gravity in space". Another common claim is that a table exerts no force on something sitting on it -- instead of "exerting a force" it's just "getting in the way".

Here is some food for thought for educators. If GPT-2 also makes sense of the world by regurgitating what it sees, perhaps this is simply the nature of learning by example, and we should accommodate for this. Perhaps it isn't so effective to give students mounds of problem sets offering clear premises and easy-to-grade answers. Unless you want your students to be GPT-2s.

[0]: https://news.ycombinator.com/item?id=21729619

[thrwaway69]

I wonder if gpt2 or similar projects can be used to make systems to train teachers. Teacher explains something and raise questions or statement and have GPT2 complete them. That way, they can learn more about students, common questions, misunderstandings, etc.

If someone knows more about what companies or tech is used for training teachers, do let me know. I am pretty interested in any vacuum in the industry and if schools pay enough for training their teachers.

[drongoking]

Arguing that physics students don't understand equations very well is a poor way to make a point about GPT-2. GPT-2 fails at a much more basic level, and that's Marcus's point.

Talk to a five year old for a while. The five-year-old's language may be crude but it shows basic concepts of a conversation, continuity, referents, basic causality, etc. GPT-2 has none of these. It regurgitates smooth language fragments because that's what it was trained on, but it exhibits no awareness that it's involved in a communication event with another being. Except possibly at cocktail parties, humans don't simply regurgitate words.

[missosoup]

Physics students generally not understanding physics isn't an isolated case. The point is that the majority of people don't really 'understand' the concepts that they talk about a lot of the time. What does it even mean to understand something? Is it binary? Is it a continuous scale?

There's no test for 'true' understanding, there's no test for awareness. There's fundamentally no way to distinguish a p-zombie from a conscious being. It's possible that p-zombies are also fundamentally incapable of distinguishing themselves from 'truly' aware beings, in which case the distinction between awareness and non-awareness is meaningless.

Trying to detect 'awareness' or 'understanding' in communications is a dead end. There is no reason to believe that a human doesn't also 'just 'regurgitate smooth language fragments because that's what it was trained on'. In fact, I see a lot of that in the professional services world. People have built entire careers by stringing plausible-sounding sentences together even though they're completely devoid of meaning if you actually try to parse them. The most interesting thing is that those people genuinely believe that they know what they're talking about. They rarely admit or believe that they lack understanding, even if it's clear to everyone around them.