[Nition]

Related to this, often I see a survey question along the lines of say, "which cup do you think would hold more water" with one that looks like it'd hold a lot more than the other. I'm never sure whether to answer with the meta-analysis of the fact that it's a survey question included in my answer.

That is, if I saw the two cups in my example naturally I'd say the obvious one holds more water, but often there'd be no point in putting it as a survey question if the obvious answer was the case. Therefore I'm almost sure the correct answer is the seemingly wrong one.

So I put that as the answer and get it right. But for whoever's running the survey, I'm probably messing up their results by meta-analysing the questions.

[tikhonj]

I don't know if this question is an example, but people intentionally include "obvious" questions in surveys to check whether people are actually paying attention to the survey. There's a name for these, I just don't recall it at the moment—which shows you I don't have any survey-design experience myself :).

[jaredsohn]

According to a Google search, "Attention check questions" sounds like the right term

[disgruntledphd2]

Also known as lie scales and have been around since at least the 50's. (Eysenck was first, I believe).