[naikus]

I'm using chatGPT 4o and it gave me the following :)

ME Alice has 40 brothers and she also has 50 sisters. How many sisters does Alice's brother have?

ChatGPT To determine how many sisters Alice's brother has, we need to use the information provided:

Alice has 40 brothers and 50 sisters. Since Alice's brother is one of the 40 brothers, he shares the same family structure as Alice. This means he has the same number of sisters as Alice.

Therefore, Alice's brother has 50 sisters.

[gwbas1c]

Alice has 50 sisters.

Alice's brother has Alice as a sister (1) + Alice's sisters (50).

Therefore Alice's brother has 51 sisters.

[happypumpkin]

First result:

Me > Alice has 40 brothers and she also has 50 sisters. How many sisters does Alice's brother have?

GPT4o (web version) > Alice has 50 sisters. Since all of Alice's brothers and sisters share the same set of siblings, each of Alice's 40 brothers also has 50 sisters. Therefore, Alice's brother has 50 sisters.

---

Step by step:

Me > Alice has 40 brothers and she also has 50 sisters. How many sisters does Alice's brother have? Think about it step by step.

GPT4o > To determine how many sisters Alice's brother has, follow these steps:

    Identify the total number of siblings: Alice has 40 brothers and 50 sisters, plus Alice herself. Thus, the total number of siblings is:
    40 (brothers)+50 (sisters)+1 (Alice)=91 siblings
    40 (brothers)+50 (sisters)+1 (Alice)=91 siblings

    Focus on Alice's brother: Alice's brother is one of the 40 brothers.

    Determine the number of sisters for Alice's brother: Each brother has the same number of sisters. Since the brothers do not count themselves as sisters, they only count the 50 sisters, excluding themselves and other brothers.

Therefore, each of Alice's brothers has: 50 sisters

So, Alice's brother has 50 sisters.

---

Thinking step by step somehow gave an even more nonsensical answer, I'm actually very surprised it didn't work when prompted to do it step by step.

[gweinberg]

I don't know why you are getting downvoted. I was disappointed the paper didn't give examples of the LLM confidently expressing its incorrect reasoning.

From a human perspective, I think there are three ways to get the answer wrong: failure to realize that Alice's brother has pretty much the same number of sisters as Alice herself, failure to realize that the brother has one additional sister, namely Alice, and failure to successfully add one to the number of sisters. But that assumes that the LLM is more or less reasoning like a human. It may well be "reasoning" more along the lines of "I've seen lots of story problems like this, the modal answer was five, I'll say five"