|
|
|
|
|
|
by Rioghasarig
2339 days ago
|
|
|
I can't agree with you here. While GPT-2 isn't good at filling in particular details (like the language someone from Boston should speak) it is astonishingly good at recognizing the _kind_ of answer that should be produced. The fact that it usually answers with a language here is a reflection of what it understands. And it behaves similarly for a whole range of different tasks. If you write a sentence that should obviously end with the name of a person it will give you the name of a person. If you write something that should be completed by the name of an event it will give you the name of an event. If you write text that should obviously be finished by the description of a person's appearence it will give you a description of a person's appearence. GPT-2's grasp of the appropriate context of a wide range of situations is nothing short of incredible. Dismissing this as "Eliza on steroids" is just short-sighted. |
|
|
I think you're anthropomorphizing a fair amount. GPT-2 has memorized huge amounts of text and can, yes, generalize the characteristics of words that fill certain slots. To say that it recognizes the kind of answer that should be produced is implying agency where there is none.
The original author is right. GPT-2 has no idea what it's talking about. Play with it any amount of time and you'll realize this. It's more than "Eliza on steroids," but the impressiveness of GPT-2 comes from the style of its language, not the substance.