[cmcaleer]

"Stochastic parrot" is a perfectly reasonable description of GPT as a technology. I use this term, and also use LLMs all the time for various tasks. It's far more descriptive than "AI". Ultimately (most) LLMs are simply next token predictors.

Understanding this and not over-anthropomorphising can help you get the most out of using LLMs and understanding where it might hallucinate. For example, the fact that it's just a stochastic parrot means that, even 2 years on, it will give the wrong answer to prompts like:

User: A man and his son are in a car accident. The man is totally fine and in good health. The man is a surgeon. The nurse asks the surgeon to operate on the son, because the surgeon is healthy and capable of doing this. The surgeon replies "I can't operate on this child. He is my son."

What happened?

ChatGPT: This riddle is a play on assumptions. The twist is that the surgeon is actually the boy's mother. The riddle relies on the common stereotype that surgeons are male, leading people to overlook the possibility that the surgeon could be the boy's mother.

[fragmede]

If you feed it:

> this is a not the normal formulation of the riddle, you won't get it right. read the problem carefully, list the constraints. describe the original riddle step by step. compare it to this riddle. what changes have been made to this from the original?

> A man and his son are in a car accident. The man is totally fine and in good health. The man is a surgeon. The nurse asks the surgeon to operate on the son, because the surgeon is healthy and capable of doing this. The surgeon replies "I can't operate on this child. He is my son." What happened?

it's able to get it right. If you ask it "why is this not the original", it's able to write an essay about why it's not.

https://chatgpt.com/share/3e98004b-58d0-4fb5-841a-336dca6037...

I found a similar thing with the river crossing problem. I was able to get chatgpt to recognize its bias with the following:

> this is a trick question, you won't get it right. read the problem carefully, list the constraints. reread the constraints that you assumed and see if they actually apply to this instance of the question.

> A farmer went to a market and purchased a wolf, a goat, and a cabbage. On his way home, the farmer came to the bank of a river and rented a boat. Luckily, the boat was large enough to carry the man and all his purchases at the same time.

> If left unattended together, the wolf would eat the goat, or the goat would eat the cabbage.

> The farmer's challenge was to carry himself and his purchases to the far bank of the river, leaving each purchase intact. How could he do it in the shortest amount of time?

Given that, chatgpt is able to get it right.

I wish I was a TA in college still and could ask a whole bunch of students to the modified questions to see how many pattern match and are stoichastic parrots.

What's the fewest words you can get chatgpt to give a right answer to either modified puzzle?

Though, what this all really says is that a lot of people are stochastic parrots, and that most people live in their own echo chamber.

[paulcole]

> User: A man and his son are in a car accident. The man is totally fine and in good health. The man is a surgeon. The nurse asks the surgeon to operate on the son, because the surgeon is healthy and capable of doing this. The surgeon replies "I can't operate on this child. He is my son."

Just for clarity’s sake, what is the right answer?

[jjw1414]

In this case, the User deliberately changed the text of a fairly well-known riddle in order to “fool” the LLM. What you really are looking for is the right question: “A man and a son get into a car accident. They are rushed to the hospital and the boy requires surgery. The surgeon looks at the boy and says "I cannot perform surgery on him, he’s my son!". Why is this the case?”. In this corrected version, there is no statement that the man is a surgeon. If you read this correct version of the riddle, ChatGPT’s answer makes sense. The LLM was tricked because the majority of the text for the User prompt (modified riddle) is very close to the correct version which is prevalent across the internet (search the web for, “I can’t operate on him, he’s my son” and you’ll receive many hits with the full text of the correct riddle). I admit that when I first read the modified version of the riddle, I assumed it was the original after having seen it so many times before.

As an aside, this riddle is commonly used to demonstrate that people assume all doctors are male based on a stereotype. However, I think the reason many people stumble on the answer is related to the exceedingly low probability that a boy in a car accident would be rushed to a hospital where his own mother (or father) is the ER surgeon. The riddle also adds in the seemingly unnecessary statement by the doctor that she can’t operate on the boy because he is her son. Why not? Possibly she would not be in the right frame of mind due to the emotional attachment, but my expectation is that her fight-or-flight would kick in and she would perform the surgery to save her son’s life. It’s a red-herring in my opinion.

[paulcole]

> For example, the fact that it's just a stochastic parrot means that, even 2 years on, it will give the wrong answer to prompts like:

>User: A man and his son are in a car accident. The man is totally fine and in good health. The man is a surgeon. The nurse asks the surgeon to operate on the son, because the surgeon is healthy and capable of doing this. The surgeon replies "I can't operate on this child. He is my son."

>What happened?

Just for clarity’s sake, what is the right answer?