|
|
|
|
|
|
by lsy
1190 days ago
|
|
|
This is a pretty fluffy paper, especially for an institution like Microsoft Research. It says it's an "early AGI" in the abstract, but elsewhere says it's merely a "step towards AGI". The basis for this is asking ChatGPT a bunch of stuff, but they don't really present an overarching framework for what questions to ask or why. The paper makes outlandish claims like "GPT-4 has common sense grounding" on the basis of its answers to these questions, but the questions don't show that the model has common sense or grounding. One of their constructed questions involves prompting the model with the equator's exact length—"precisely 24,901 miles"—and then being astonished that the model predicts that you're on the equator ("Equator" being the first result on Wikipedia for the search term "24,901"). It's also the case that while GPT-4 can say a bear at the north pole is "white", it has no way of knowing what "white", or "bear", or "north" actually represent. Are there folks out there doing rigorous research on these topics, who have a framework for developing tests of actual understanding? |
|
|
This is a preposterous claim that you could easily disprove within a few minutes of using it.