[visarga]

Actually, if you ask GPT3 to explain step by step how it arrived to the conclusion, it works pretty well and even has higher accuracy.

[mr_toad]

GPT is making it up.

It’s cobbling together an explanation from texts that explain how to solve a problem. It’s like a student who copies an answer, then copies the explanation as well.

[visarga]

Chain-of-thought is not the same with direct prompting. Step by step is better.

> Experiments on three large language models show that chain of thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks. The empirical gains can be striking.

https://arxiv.org/abs/2201.11903

[l33t233372]

But there’s no reason to believe that is actually how the result was generated.

The result was generated via a deep neural network, not by whatever explanation or reasoning it prints out when asked.