[geuis]

Mixtral 8x7b-32768 got it on the first try:

  Sally has 1 sister.

  Here's the logic behind this:

  1. We know that Sally has 3 brothers.
  2. Then we are told that each brother has 2 sisters.
  3. Since Sally is one of the children in the family, there must be another sister besides Sally.
  4. Therefore, Sally has 1 sister.

  The number of sisters Sally has is not affected by the number of brothers she has, so the information about her brothers having 2 sisters is not relevant to finding the number of sisters Sally has.

[Jackson__]

I'm amazed mistral is still doing the inverse chain of thought reasoning by default, even with their new large model.

This causes it to get the question wrong for me, when testing, and only if I manually prompt normal CoT does it get it right.

Is there any papers showing a merit to this approach? It seems extremely counter-intuitive.

[lulznews]

It got the answer but the reasoning in the last paragraph is wrong ..

[walkieer]

This is impressive because I forgot Sally counts as a sister too... and the reasoning for the most part is correct and really easy to understand.