[V__]

Every time I come across this or similar examples, the phrasing always tricks one into thinking "A therefore B" and "B therefore A". As soon as it's clear that is not the case, the problems become relatively easy.

I would really like to see two test groups given this test and one gets the aforementioned warning. I believe this test doesn't really test logical thinking, but our biases about what we think the rules are implying.

[nicoburns]

> I believe this test doesn't really test logical thinking, but our biases about what we think the rules are implying.

The most interesting thing this brings up is a bimodal distribution of answers between those that take a "quick intuitive answer" approach and those that a "stop and think through it carefully" approach. This can be influenced in towards either method by priming the testee as you suggest. But the fascinating thing is that if you don't do this, you find that each individual has an innate preference towards one or other method.

This is one of the main subjects of Daniel Kahneman's book "Thinking Fast and Slow" (see also: "Maps of Bounded Rationality" if you'd rather read something journal article sized), and along with introversion/extraversion constitutes one of the primary axis of the MBTI.

[V__]

Funny you bring this up. Last time I came upon this test, I took the time and solved it correctly. Today I was watching YouTube in another tab and just clicked through it and got it totally wrong.

Thank you for the MBTI reference, will check it out.

[mcphage]

I think the question is phrased pretty explicitly to be of the form "If A then B". Are you saying that they're being misleading? Or are you saying that the normal english meaning of "If A then B" is "A therefore B and B therefore A"?

The later might be the case—"or" is another case where the logical connective and normal interpretation differ; the logical "or" is inclusive, but the standard english meaning is exclusive. Then the fact that people are more likely to get question 3 correct is due to the fact that we're more familiar with the situation, and so already know that "If A then B" really is the logical connective.

[bberenberg]

It's not asking you to test the card alone. It's asking to test the rule as it applies to the card. You have to check ABBA or you're not actually applying the rule to the card. This seems like QA done by an engineer vs QA done by someone who cares about QA. The "right" answer here is wrong.

[mcphage]

> The "right" answer here is wrong.

I assume by "right", you mean the answer given? What do you think the correct answer is?

[bberenberg]

For example, the site argues that for the first test, you don't need to flip the yellow card.

Your goal, is to test the application of the rule, not, the card itself. This is specified at:

"You have to ensure that cards have been produced in accordance with the following rule"

The rule is "If a card has a circle on one side, then it has the colour yellow on the other side."

Given a card that is yellow on the other side, you must flip it to confirm that the first half of the statement is true. Why? Because the challenge isn't to test the card, it's to test application of the rule. You can't simply assume that the rule has always worked, otherwise there is no need to test.

A square on one side, with a yellow flip side would violate the rule and thus show a defect.

I get that it's going to be a linguistics argument all the way down> For me, this is a perfect example of "works for me" engineer response vs QA doing actual testing.

[mcphage]

How would you phrase a rule that adheres to the logical if-then connective?

[V__]

I don't think they are intentional misleading. It's just that most people don't read correctly stated logical statements often and might interpret more meaning into the sentence than was intended.

[gweinberg]

The problem is, if you read the rule as "if and only if", then you always have to look at everything, right? But I I think a lot of people who get the answer wrong only apply the only if logic one way. That is, if the rule is "a circle on one side must be yellow on the other", they'll check the yellow side to make sure it has a circle (even though it doesn't have to), but they won't check the square to make sure it isn't yellow. They "understand" that a square on one side and yellow on the other is allowed when they see a square, but not when the see a yellow.

[abeppu]

> I believe this test doesn't really test logical thinking, but our biases about what we think the rules are implying.

I agree, and I think for this reason the ordering of the questions is important. I think people correctly understand the rule about drinking, and handling that question first might set people up to correctly interpret the other rules.