[int_19h]

Nobody can definitely answer this question because we don't know what exactly is going on inside the model of that size. We can only speculate based on the observed behavior.

But in this case, I didn't imply that it's "reasoning beyond the domain of language", in a sense that language is exactly what it uses to reason. If you force it to perform tasks without intermediate or final outputs that are meaningful text, the result is far worse. Conversely, if you tell it to "think out loud", the results are significantly better for most tasks. Here's one example from GPT-4 where the "thinking" effectively becomes a self-prompt for the corresponding SQL query: https://gist.github.com/int19h/4f5b98bcb9fab124d308efc19e530....

Or here's an even more interesting example where GPT-4 does this kind of "thinking out loud" unprompted: https://gist.github.com/int19h/8251bd00b7a4858a69cf3922ae674...

I think the real point of disagreement is whether this constitutes actual reasoning or "merely completing tokens". If you showed the transcript of a chat with GPT-4 solving a multi-step task to a random person off the street, I have no doubt that they'd describe it as reasoning. Beyond that, one can pick the definition of "reason" that best fits their interpretation - there is no shortage of them, just as there is no shortage of definitions for "intelligence", "consciousness" etc.