[rng_civ]

Here's my abitrary line in the sand: if you give the prompt to a human, they could give a similar reply, but the prompt would also trigger other reactions such as:

* Who's Daisy?

* Why would Daisy do that?

* Daisy is rude.

etc. that imply the existence of some sort of abstract object on which relations and other facts can be plugged into. For me, the existence of that abstract object is "reasoning."

We do not know if GPT is capable of forming abstract objects in its network, and I do not think it is reasonable to infer that from its text output. In my non-expert opinion, it seems possible that the output can be achieved via knowledge regurgitation through the use of sentiment analysis, word correlations, and grammar classification.

So in this framing, it's not reasoning about Daisy nor hallucinating facts. It's regurgitating knowledge about the relationship between sentiment, words, and grammar. (An interesting experiment to run would be to change 'Daisy' to a random noun or even nonsense tokens to see what would happen).

You might argue that the ability to mechanically model that relationship counts as reasoning, and that's a stance I won't outright dismiss. However, it does seem strictly less powerful that mechanically modeling on top of abstract objects.