Hacker News new | ask | show | jobs
by SaltyLemonZest 2151 days ago
I'm not convinced that "capable of reasoning, but not consistently" is a meaningful claim. The examples seem to primarily consist of people spending hours trying things, until eventually GPT-3 outputs a chunk of reasoning they could personally do in seconds. Does that mean that GPT-3 is doing the reasoning, or does it mean that GPT-3 is an English-based lookup table and they managed to find a clever sequence of search keys?

The fact that there could be reasoning going on is certainly exciting by itself. But I don't think it's fair to call it obvious without a compact specification for how to make GPT-3 perform a general class of reasoning. Less "here's a script to make it output stuff about balanced parens", more "here's a strategy to teach it most basic string manipulations".
2 comments
drdeca 2151 days ago
> I'm not convinced that "capable of reasoning, but not consistently" is a meaningful claim.

Suppose an entity will consistently do reasoning well, but only when the humidity and temperature are each in a quite narrow range. It seems like it makes sense to say that such an entity is capable of reasoning. Now, suppose we don't know that the conditions for it to do reasoning well are that the humidity and temperature are in that range, we just know that sometimes it looks like it does, sometimes it looks like it doesn't (and maybe we aren't yet sure if it seeming to reason is just an illusion in the way you describe).

I think in such a situation, it would be accurate to say that it can reason, but we haven't yet found a way to make it do so consistently.

So, I think the statement that it is "capable of reasoning, but not consistently" is a meaningful statement.

However, whether it is an accurate statement is a very different question, and one which I am not claiming an answer to.

link
TomMarius 2151 days ago
Have you seen the database prompt?

https://www.gwern.net/GPT-3#the-database-prompt

link
SaltyLemonZest 2150 days ago
As I mentioned, I don't think any single prompt can demonstrate the presence of true reasoning. If the prompt isn't shown to broadly generalize, it might just be doing a text match to something that was said before on the depths of the internet. You can see this in the next section; Kevin Lacker gets GPT-3 to demonstrate it knows some basic trivia questions, but it "knows" any prompt with the same textual structure as a basic trivia question, even if the prompt is nonsense. This strongly suggests that it's parsing out key words and doing a lookup on them rather than accessing a consistent internal model.
link