|
|
|
|
|
|
by wilg
2339 days ago
|
|
|
I don't agree with the conclusion here. It's all about the input data. GPT-2 is trained on words people actually write on the internet, which is an inherently incomplete dataset. It leaves out all the other information an "intelligence" knows about the world. We know what sources are authoritative, we know the context of words from the visual appearance of the page, and we connect it all with data from our past experiences, school, work, friends, our interaction with the world. Among a million other ways we get data. How would GPT-2 determine most facts from the input dataset? If the only thing you knew was all the text on the internet, with zero other context, you'd have no way of knowing what is "true", or why that concept is important, or anything else. I bet you'd behave just like GPT-2. It's a robot that is really good at writing, because that is all it knows. I think it doesn't know anything about how to make sense on a macro scale because I don't think the input data contains that information. It seems to do well when the input data contains relevant information. |
|
|
It sure is. Go to the site [1] and paste in anything from an Internet rant. It does a really good job of autocompleting rants.
At last, high-quality artificial stupidity.
The other extreme is the MIT question-answering system [2]. Or Wolfram Alpha. Just the facts.
[1] https://talktotransformer.com/ [2] http://start.csail.mit.edu/index.php