[travisjungroth]

You don’t get the horse analogy. The horse isn’t trained to count to eight. The horse is trained to stomp continuously. The performer yells “stop” at eight. It’s a carnival trick and a form of cherry-picking. The commenter was saying that ChatGPT doesn’t have the ability to solve these problems at all and people are just choosing the randomly correct answers. I think this is obviously not true.

[kazinator]

The purpose of the horse analogy wasn't so much to equate ChatGPT to a horse that just stomps, but the behavior of the humans around it who believe that it can count. This can be the naive trainer too, who doesn't recognize that he's subconsciously cuing the horse. He has a confirmation bias whereby he or she rejects any evidence refuting the hypothesis that the horse cannot count.

There is a difference between solving and problem and presenting a solution, and that difference doesn't hinge on whether the solution is correct.

[travisjungroth]

ChatGPT is making me realize how completely differently I see the concepts of knowledge and ability from so many people.